Thomas Boch (CDS, Observatoire de Strasbourg), Pineau (CDS), Derriere (CDS)
Abstract
The CDS has released a cross-match service allowing astronomers to cross-identify sources between very large catalogues (up to 1 billion rows) or between a user-uploaded list of positions and a large catalogue. This service has been built on top of the methods described at ADASS 2010 by Pineau et al., which are based on a dedicated binary table file format and the HEALPix pixelisation associated to specialized KD-trees.
Popular cross-identifications, such as SDSS vs. 2MASS, are pre-computed in order to accelerate these queries.
We will present the architecture of the service, whose core relies on an implementation of the UWS (Universal Worker Service) pattern, a Virtual Observatory standard for asynchronous, stateful services.
Scalability is achieved thanks to a distribution of the different crossmatch jobs on a set of workers, which are also accessed through a UWS interface.
Users interact with the service through a Web interface.
Submission of cross-match jobs through command-line tools are also possible.
Results of the cross-match are stored on a user personal storage space - backed up by iRODS - which is already in production since 2010 for the CDS Portal.
In our poster, we will show the architecture of the service, present the file formats strategy we used to integrate VizieR tables, and discuss our hardware choice, in term of RAM and hard disk drive. Poster in PDF format
Paper ID: P013
Poster Instructions
|
