The NGAS Express Synchronization Tool, ngasXSyncTool, is used to carry out bulk data migration or cloning within the NGAS System in an efficient manner. The tool works in an cluster oriented manner meaning that the target for the synchronization is an NGAS Cluster, which name space is used to derive which files to clone and to which target nodes the cloning can be done. An NGAS Cluster is defined by a set of NGAS Nodes, which have the same Cluster Name in the NGAS DB (ngas_hosts.cluster_name). When deciding which files to clone to the target cluster, the basic criteria applied by the tool is that there should be one and only one valid file in the target cluster name space. Determinating if a files is safely stored in the target cluster is done in a secure way, by sending a CHECKFILE Command to the target cluster, which will result in an explicit test of the consistency and availability of the file in the target cluster name space. Files can either be specified as single files in a standard NGAS File List, or the ID of a volume ID (Disk ID) can be given and all files hosted on the volume will be considered for cloning. It should be specified with the --streams parameter how many parallel streams the tool should control. It is important to find the 'appropriate number of streams'. Specifying too many streams when sync'ing a HDD-based volume, may reduce the performance haevily since the read/write head of the HDD will be offsetting continuesly. On the other hand, it is clearly more efficient to have a number of parallel streams sync'ing the data, compared to only one stream, executing the batch sequentially. After the tool has been executed on a set of files, it can also be used to verify that the cloning was successful and can be used to correct possible failing file synchronizations by simply executing it once more. The tool implements persistency, such that if interrupted in the middle of the processing of a batch, it will resume from the point where it was interrupted. Should the files however, be lost which provides this persistency, the tool will nonetheless be capable of resuming the processing from where it was interrupted, after checking first all files in the set that have been cloned successfully. A report of the actions carried out by the tool is send out to a specified list of recipients, given either on the command line or in the ~/.ngas resource file. This report will contain a summary of the execution and the entire list of files that were processed and an indication if the synchronization was successful or not. The tool can be instructed to send out intermediate notification reports via email to the specified list of recipients. These intermediate reports will contain a summary of what has been cloned already, and how much is missing. The tool uses a working directory under which all house keeping files will be stored. The default value for the working directory is '/tmp', but it is recommended to create a proper working directory for the tool, typically under the account of the user executing the tool. For each synchronization batch, the tool generates a Session ID for the session. All files pertinent to one session, will be stored in a subdirectory in the working directory: /Session ID> The Session ID is generated as follow: 1. When synchronization volumes: The Session ID is generated as the concatenation of the hostname and the NGAS Slot ID for the NGAS Node hosting the volume to synchronize. 2. When synchronizing a list of files: The Session ID is generated as the MD5 checksum of the contents of the file list. Files and session directories in the working directory will be kept 30 days after creation, independently of if the synchronization batch was successfully terminated or not. The input parameters defined for the tool are: --help [0] (OPTIONAL): Print out man-page and exit. --debug [0] (OPTIONAL): Run in debug mode. --version [0] (OPTIONAL): Print out version. --verbose= [0] (OPTIONAL): Switch verbose mode logging on ([0; 5]). --logFile= [None] (OPTIONAL): Log file into which info will be logged during execution. --logLevel= [0] (OPTIONAL): Level applied when logging into the specified log file. --notifEmail= [None] (OPTIONAL): Comma separated list of email recipients which will receive status reports in connection with the tool execution. --accessCode= [None] (OPTIONAL): Access code to access the NGAS system with the NGAS Utilities. --force0 [0] (OPTIONAL): Force execution of tool. --disk-id= [None] (OPTIONAL): ID for the volume to synchronize. If not given, the --fileList parameter must be used to specify the source files. --file-list= [None] (OPTIONAL): List containing references to files to consider for synchronization. Each line must specify: . Disk ID may be omitted to indicate that any file is chosen as source file. If not given, the --diskId parameter must be used to specify the source files. --target-cluster= [None] (MANDATORY): Name of NGAS Cluster to which the files will be synchronized. --streams= [None] (MANDATORY): Number of streams to execute in parallel. --working-dir= [/tmp] (OPTIONAL): Working directory of the tool. The tool will create a directory '/NGAS_XSYNC_TOOL' directory, where it will store its house-keeping files. --notif-email= [None] (OPTIONAL): Comma separated list of email addresses. --intermediate-notif= [None] (OPTIONAL): Time in seconds for which to send out an intermediate status report for the processing of the synchronization batch. Copyright (c) European Southern Observatory All rights reserved.