Science Archive and the Digital Universe
About 12 Terabytes (TB) of public data are distributed per year through the ESO archive, following about 10 000 web requests. Additionally, more than 2 000 CDs and DVDs of proprietary data are sent out annually to their respective Principal Investigators for observations made on their behalf (service mode). The current total archive holding is about 65 TB, with an input rate of about 15 TB per year. This will soon drastically increase by a factor of 10 or so as the Visible and Infrared Survey Telescope for Astronomy (VISTA) with its near infrared camera will alone produce about 150 TB of data each year.
ESO's enterprise-class database servers are coordinated between Germany and Chile, and their technology and complexity rivals that of major commercial enterprises such as the international banking community.
The Digital Universe
Major breakthroughs in telescope, detector, and computer technology now allow astronomical surveys to produce massive amounts of images, spectra, and catalogues. These datasets cover the sky at all wavelengths from gamma- and X-rays, through optical and infrared, to radio waves.
Astronomers are developing ways to do new science, by making the huge amount of data in this 'digital Universe' easily accessible. These techniques use the GRID paradigm of distributed computing, with seamless and transparent access to the data through 'Virtual Observatories'.
Just as a physical observatory has telescopes, each with unique astronomical instruments, a Virtual Observatory consists of data centres, each with unique collections of astronomical data, soft-ware systems and processing capabilities.
This global, community-based initiative is being developed world-wide under the auspices of the International Virtual Observatory Alliance (IVOA) and in Europe under the framework of the EURO-VO project.
Virtual Observatories have already proved their effectiveness, for example by discovering 31 new optically faint, obscured quasar candidates in the existing Great Observatories Origins Deep Survey (GOODS) fields, quadrupling the number previously found. The discovery means that surveys of powerful supermassive black holes have so far underestimated their numbers by at least a factor of two, and possibly by up to a factor of five (ESO Press Release eso0418).