make printable	new:	Note:
	v1.10: - de-commissioning of ngamsCClient v1.9: - handling of both the DFOS-v1 and the qcFlow naming schemes for MCAL and PHDR download v1.8: - new option -i for IDPs	The tool is the central download engine for DFOS. databases ngas for file availability; qc_products for product file names and for ngams server names dfos tools: ngamsCClient and dpQuery (dfs), requires NGAS access; hotfly output: hdr, fits files upload/download: wget from archive.eso.org

ngasClient

Description

This is the general dfos tool for downloading raw or product files from NGAS, including IDPs. You can download fits files or their headers only. Contrary to dataLoader, it downloads file by file.

It uses the Linux command curl (new with v1.10) for raw data (and tar ball) download, and the DFS tools dpQuery for product data download and hotfly for header downloads and updates. To uncompress tile-compressed files (like VCAM files), you also need the DFS tool fitsunpack. In either case, you need to specify the file name (ARCFILE for raw files, product file name as given by renameProducts).

Header download. The tool is able to download FTU'ed headers, and updates headers of downloaded fits files (only if necessary: modification_timestamp is later than ingestion_timestamp). The update of the FITS header is made through hotfly (headers on the fly). It uses a software called FTU (fits translation utility). Find more details here. It is also enabled to directly download headers of product files, without downloading the fits file in the first place.

There is another option to download headers, -W: these are downloaded from the cgi on the archive web page (http://archive.eso.org/hdr?DpId=) using wget. This is a slow process and kept for backwards compatibility only. See our hotfly page for a more complete compilation of header download processes and tools.

Note:

- the hotfly update is executed on the full header (primary plus extensions)
- use option -c for downloading any product fits file (CALIB or historical SCIENCE or historical ANCILLARY; no IDPs!)
- use option -P for downloading any product header file
- the raw files (option -f) and product files (option -c) are downloaded only if the specified file does not exist in $DFO_RAW_DIR / $DFO_CAL_DIR (unless forced by option -F).

How to use

Type ngasClient -h for on-line help, ngasClient -v for the version number.

Type

ngasClient -f VISIR.2026-02-12T13:51:49.000.fits

to download a raw file and update its header (if a corresponding hdr file exists in the same directory, it is deleted),

ngasClient -c VI_MBIA_260304C_1x1.fits

to download a master calibration file (if a corresponding hdr file exists in the same directory, it is deleted),

ngasClient -i GI_SIDP_1179308_2015-07-30T03:08:45.415_F9_Med1_L614.2_o3.fits

to download an IDP (science product file; any format: fits/gif/txt),

ngasClient -H GIRAF.2026-06-06T03:34:34.345.hdr (or .fits)

to download the header of a raw file, and

ngasClient -P GI_SFLX_2019-12-31T04:31:12.561.fits

to download the header of a product file. Note: although you want to download a header, you must specify the fits file name since this is used in a query, and it is not possible to safely predict the name of a product fits file from its hdr name (it could have extension .fits or .tfits!).

You can also use ngasClient -W FEROS.2026-06-06T03:34:34.345.hdr to download a header file but this is slow and kept mainly for backwards compatibility.

The file is downloaded into the current directory. If the file is delivered with extension .Z (compressed), it is uncompressed. Tile-compressed files (with extension .fz) are also uncompressed.

If called with the option -F (Force), the file is always downloaded, no matter if it pre-exists or not.

For IDP download, you can specify any file name archived in phase3, no matter if fits, text or graphical file, and no matter if a genuine IDP or an ancillary file. In order to get a complete dataset (IDP plus all ancillary files), you need to specify all names. (If unknown, download the IDP first, then read the ancillary file names from the ASSONn keys and download them as well.)

Installation

Use dfosInstall.

Configuration file

With version 1.10, a new configuration file is needed: $HOME/.netrc. It contains the credentials for the QCUSER User Portal account which enables the download of proprietary archive files. Its content is:

machine dataportal.eso.org login QCUSER password PWORD

With version 1.10, the option -U and the configuration file ngasClient.cfg have been de-commissioned.

Operational hints