Skycat - Notes on Writing an AstroCatalog Server

CCLRC / RUTHERFORD APPLETON LABORATORY
Particle Physics & Astronomy Research Council
Starlink Project

Clive Davenhall
Institute for Astronomy, University of Edinburgh & Starlink
27th October 1996

Notes on Writing an AstroCatalog Server
(DRAFT)
Version 1.0

These notes are an incomplete draft. Queries, asides and things I'm unsure about are shown in a sans serif font like this.

Introduction
Accessing a New Server with the AstroCatalog Library
Query parameters
- Example
- Generalised queries
Tab-separated list
- Example
Hints
Related Documents
- Specific references
- General references

Introduction

The ESO AstroCatalog library allows applications to access remote databases, catalogues and archives across the internet. It is used by the ESO skycat image display tool and various other applications. In order to make a database accessible to the AstroCatalog library a `server' must be provided for it. This server will accept remote queries sent in a standardised format, interrogate the database and return a table of results corresponding to the objects in the database which satisfied the query. These notes are intended to assist in writing such a server. They should be used in conjunction with listings for one or more existing servers.

The communication between the remote application and the server uses the Hyper-Text Transfer Protocol (HTTP) developed for the World Wide Web. The servers are, strictly speaking, gateways using the Common Gateway Interface (CGI). One way of thinking of the remote application is as a very specialised Web browser. One consequence of this approach is that if a site is to provide an AstroCatalog server it must also be running a Web server. The HTTP and CGI protocols are, of course, enormously flexible and there are additional protocols and conventions for communicating between an AstroCatalog application and server. These protocols and conventions are described in subsequent sections. The format for queries is described in Section 3 and that for the returned results in Section 4.

Though the protocols for communicating between an AstroCatalog application and server are (reasonably) well defined, there are an unlimited number of ways in which a server could be written to accept standard queries and return standard results. Thus, these notes cannot prescribe how a server must be written, but merely suggest some alternatives. An AstroCatalog server is a CGI script and CGI scripts are usually (but not necessarily) written in Perl. For the rest of these notes I shall assume that the server is to be implemented as a Perl script. The entire server could be written in Perl, with the database being accessed directly from the Perl script. However, it is more likely that the Perl script will invoke a Database Management System (DBMS) or special-purpose program to interrogate the database.

This document assumes that you have some familiarity with the AstroCatalog library or, at least, skycat. The AstroCatalog library is documented in Astronomical Catalog Library Interface Specification by Allan Brighton. It also assumes that you are familiar with HTTP, writing CGI scripts and Perl. There are numerous books describing the HTTP and CGI protocols. The HTML Source Book by Ian Graham seems to be a particularly good one. Similarly there are various books on Perl. Learning Perl by Randall Schwartz and Programming Perl by Larry Wall and Randall Schwartz are comprehensive and easy to use.

Accessing a New Server with the AstroCatalog Library

It might seem perverse to describe how to access a new server with the AstroCatalog library before describing how to write the server. However, access to the new server will often be the first thing that you set up, so that you can try it out during development and debugging.

The servers for catalogues, databases and archives are identified to the AstroCatalog library using a syntax of the form:

name@location

where name is an abbreviation for the name of the catalogue and location an abbreviation for the institution where it is held. By convention both name and location are usually quite short. For example:

ppm@eso

is the abbreviation for the version of Bastian and Roeser's accents? Positions and Proper Motion catalogue available at ESO.

A list of all the catalogues, databases and archives which the AstroCatalog library can access is held in a configuration file. You must take a copy of this configuration file, add details for your new server and then instruct your copy of the AstroCatalog library to access this modified file.

By default the AstroCatalog library uses a configuration file kept at ESO. You can retrieve a copy of it from URL:

http://archive.eso.org/skycat/skycat2.0.cfg

To access your own copy of this file you must place it in a directory accessible to your local HTTP server (that is, one which is `visible to the Web'). Then set environment variable SKYCAT_CONFIG to a URL which points to this file.

You must now add an entry for your catalogue to this file. The file is simply a text file which can be modified with a text editor. It contains extensive comments which should be read in conjunction with these notes.

For example, the entry for the PPM catalogue is:

serv_type: catalog
long_name: PPM at ESO
short_name: ppm@eso
url: http://archive.eso.org/skycat/servers/ppm-server...
symbol: mag circle 15-$mag

The purposes of the various items are as follows.

serv_type:The alternatives are: catalog, archive, namesvr or imagesvr. catalog is the simplest and most common option; it used for simple catalogues and tables. A serv_type of catalog is assumed in the rest of these notes. See Section 2 of the Astronomical Catalog Library Interface Specification for a discussion of the alternatives.

long_name:A name or short (one-line) description of the catalogue. It is intended to be read by humans rather than interpretted by computer.short_name:The name used to identify the server to the AstroCatalog library.url:The URL used to access the server. Following the usual conventions for a CGI gateway it consists of the directory specification and file name of the server followed by parameters passed to the server to define the query (see Section 3).symbol:Defines the plotting symbol to be used; see the comments in the configuration file for details.

symbol is optional; the other items are mandatory.

Finally, if you want your server to become generally available then remember to send either the details or your modified configuration file to the skycat/AstroCatalog library team at ESO.

Query parameters

The parameters at the end of the server URL pass values which define the query which objects in the catalogue must satisfy if they are to be selected. The types of queries which are currently supported are mostly concerned with selecting objects within a specified region of sky. The region may be either:

`circular' - within a given angular distance of a specified central point,
`annular' - within an annulus defined by a minimum and maximum angular distance from a specified central point,
`rectangular' - within a region bounded by specified great circles of Right Ascension and parallels of Declination.

The query parameters are as follows.

%raRight Ascension of the central position specified as sexagesimal hours check in J2000 coordinates.

%decDeclination of the central position specified as sexagesimal degrees in J2000 coordinates.%r1The minimum radius for an annular query, in minutes of arc. For a circular query it should be set 0.0.%r2The maximum radius for an annular query or the radius for a circular query, in minutes of arc.%wThe width or Right Ascension range of a rectangular query, in minutes of arc.%hThe height or Declination range of a rectangular query, in minutes of arc.%m1The minimum (brightest) magnitude for an object to be selected,%m2The maximum (faintest) magnitude for an object to be selected,%nThe maximum number of objects to be selected.%colsA list of columns to return. The list comprises a set of column names separated by commas, for example:

col1,col2,...coln

%idID field of item to return (if supported).%mime-typeValue for HTTP mime-type field.

It is not always necessary (or possible) for a server to support all sorts of queries. For example, the minimum and maximum magnitude has no meaning in the case of the UK Schmidt Plate Catalogue.

Example

The query parameters to select objects within 120 minutes of arc of 10:00:00, +30:00:00 in the PPM catalogue was:

http://archive.eso.org/skycat/servers/ppm-server
?ra=10:00:0.00&dec=+30:00:0.00&radius=120&nout=101

Generalised queries

The query parameters used by the AstroCatalog library are special cases of a proposed general format for exchanging information between remote astronomical information services. A working document describing this proposal is available at URL:

http://vizier.u-strasbg.fr/doc/asu.html

Note that there seem to be differences between the query parameters currently implemented and the proposal.

Tab-separated list

The server must return the list of selected objects as a tab-separated list. The tab-separted list format is described in Section 2.3 of Astronomical Catalog Library Interface Specification. Briefly, it is a file or other stream of bytes, in which the values for individual fields in the list are separated by a tab character (ASCII character 9). A file in tab-separated list format can be modified with a text editor. The details of the format are as follows.

The first line of a tab-separated list must be a header. Usually this header will be:

Content-type: text/plain
The next line gives the names of all the columns in the list, separated by tabs.
The following line is a list of dashes (and tabs) which indicate that the table of values will follow immediately.
The table follows, with one row (or object) per line.
Within each line the fields corresponding to each column occur in the same order as the column names (given in 2 above) and are separated by tabs.
The following two additional constraints apply:
- the first column must be a name or identifier,
- the second and third columns must be respectively be the Right Ascension and Declination in decimal degrees and J2000 coordinates.

The optional special columns MORE and PREVIEW can be used within information and image servers (see Section ???)

Example

A tab-separated list corresponding to a some columns in a subset of the PPM catalogue is listed below.

Content-type: text/plain

Id ra dec mag
-- -- --- ---
+29 1956 148.028620833333 29.2643666666667 9.2
+31 2059 148.081295833333 30.3859694444444 9.5
+29 1958p 148.092170833333 28.5661469444444 9.8
+32 1947 148.121254166667 31.5426083333333 8.6
+29 1959 148.179058333333 29.0041111111111 9.6

Hints

When developing a server it is often useful to see the query that has been sent by the remote application. If you are using skycat as the remote application then the last query sent to the server is included in the skycat log, which is file:

~/.skycat/log

If you are using some application of your own rather than skycat then the AstroCatalog library is easily modified to echo the query when it sends it. Proceed as follows.
1. Locate and edit file AstroCatalog.C.
2. Locate the lines:
  
  // send the query
  result_buf = http_.get(buf, nlines);
3. Print out the value of buf by inserting something like:
  
  cout << buf << endl;
  
  between these two lines.
4. Regenerate the execution module for your application.
Special columns MORE and PREVIEW.
plus others?

ESO — Reaching New Heights in Astronomy

Skycat - Notes on Writing an AstroCatalog Server

Contents

Introduction

Accessing a New Server with the AstroCatalog Library

Query parameters

Example

Generalised queries

Tab-separated list

Example

Hints

Related Documents

Specific references

General references