README version 06/02/04 EIS/MVM SOFTWARE RELEASE VERSION 1.0.1; 2004-02-06 This is the second release (version 1.0.1) of the stand-alone EIS/MVM image reduction system. The present product release should continue to be considered as a work in progress. Users are encouraged to send comments, suggestions, verification results and requested to report any errors/features found with the released software via electronic mail to: eis-help@eso.org indicating explictly the software version in use and date of release (e.g. Version: 1.0.1, date of release: 6 February 2004). NOTES ON VERSION 1.0.1 (PRESENT) RELEASE WITH RESPECT TO PREVIOUS SOFTWARE RELEASE (VERSION 1.0; 2003-12-17) This release completely supersedes the data release of the EIS/MVM software release 1.0 performed on 17 December 2003. Differences exist between the two releases. In the present release, compilation errors and miscellaneous bugs reported by users have been resolved, as reported in the following log of (technical) changes: * mvmjitter.cc MvmJitter::get_mvmamer add in xml log the noise of images node . Futur harmonization in node name should occur. * mvmamer.cc MvmAmmer object: add option Match_min_match_amer to specify minimum number of anchor for the astrometric solution. This notion asserts a minimum absolute number of matches. * config/reduction-mode.xml update definition for jitter-base match-search-relative set to 0.5 match-min-match-relative set to 0.1 match-min-match-amer set to 4 * mvmjitter.cc MvmJitter::jitter_norm bug fix in trim_value computation. The trim-min-image option is now working properly. * cube.cc cube::set_max_image bug fix with _max_image. * scr/src_eis/Makefile set CC=g++ instead of gcc for gcc > 3 compatibility from Thomas Erben. * add_mosaic.cc add option --cosmic to switch on the cosmic rejection. * mosaic.cc mosaic.h add option to select sigma-cliping rejection. * normcalib.cc bug fix in stat computation that make the fluxscale computation wrong. * rbspec.cc add option to specify the min number of RB This help for multi-chip instrument in order to remove residual cosmic rays between chips. * config/reduction-mode.xml change definition jitter-base such as the sigma-cliping is done at 2.5 sigma before stacking 3 or more images. * mvmjitter.cc triming is done with the context map instead of computed from the weight map. This give a true triming. Option trim-sensitive disappeared. * deflat.cc new option flag-remove-bad-column to detect bad aligned pixels. The algorithm is based on minimum in gray level with direction x and y. This algo remove bad columns not observed in bias and flat. Note that the remaining bad columns could be masked out from dark. But no dark are observed in optic in general. This algo should more to mvmcalib in the future. This algo is quite heavy and add 10% of the total time to reduce an image. * fringing.cc the detected tracks of the first fringe estimations are now removed also to the original data. This prevent bad fringing estimation when a track is present. * config/reduction-mode.xml finishing clean up of old options. new definitions common for most Reduction Mode. * rbspec.c implementation of the new step to automaticly create RBs from slipted images from their object name. Since this present EIS/MVM software release incorporates improvements to completely supersede the first release, to the best of our knowledge the natural recommendation would be that the most recently available, updated release be adopted whenever possible. There has been no difference to the content of the user manual between release 1.0 performed on 17 December 2003 and the present release. INTRODUCTION Since June 2002 the EIS (ESO Imaging Survey) and the PSG (Public Survey Group) teams of DMD (Data Management Division) of ESO have been working on the development of an end-to-end, integrated, high-throughput software system. The EIS survey system benefits from the existing ESO data flow system (DFS) and archive infrastructure and consists of several elements which include: - a Python wrapper responsible for the administration of the entire system consisting of over 250,000 lines of code - a specially designed C++ based image processing system, consisting of over 100,000 lines of code, to reduce optical/infrared images from single- and multi-chip instruments, referred to hereafter as the EIS/MVM (Multi-resolution-Vision-Model) system (see BACKGROUND) - over 100 tailored tables in "Sybase" database - a WEB service based on the internal database and making extensive use of XML (extensible mark-up language) and SVG (scalable vector graphics) technologies and many other features that will be presented elsewhere. While developed primarily to support public surveys the EIS survey system can also be used as a front-end to the science archive as well as to support individual end-users. Some of the highlights of the system have been presented by da Costa (2003 a, b), in the Workshop of "Large Programmes and Public Surveys", in Garching (May 2003) and in demonstrations presented to the Survey Working Group for Public Surveys. While both Python and C code are open source, the integrated nature of the survey system prevents its distribution as a whole. However, a stand-alone version of the EIS/MVM image reduction system is being made publicly available in the present distribution. It is important to note that this system has been extensively tested using SOFI and ISAAC infrared data as well as optical multi-chip data from high- and low-galactic latitude fields taken with the wide-field imager (WFI) at the 2.2m MPG/ESO telescope at La Silla. These tests, carried out by the EIS/PSG team as well as other users, have demonstrated that the system leads to cosmetically enhanced images, including the automatic removal of satellite tracks, robust astrometric calibration and reliable relative photometric measurements. The system is also remarkably efficient as attested by benchmarks carried out on different platforms. For instance, the system reduces one WFI image per minute (or about 1 Mpix/sec) using a 4 CPU INTEL Pentium 4 machine. CONTENTS Herewith please find a "tarball" for the distribution of the EIS/MVM image reduction software. The tarball contains: - A draft version of the EIS/MVM Image Pipeline User's Manual (date) - The source code and the required XML files of the most recent version of the pipeline - This README file prepared by the EIS/PSG team - An INSTALL file describing the installation procedure BACKGROUND The EIS/MVM image processing pipeline is a software package developed by B. Vandame, funded by the EIS project at ESO. The development of the pipeline started in June 2000 using as the basis a C++ based library known as the Multi-resolution-Vision-Model (MVM) developed by B. Vandame and others at the Nice observatory under the supervision of A. Bijoui. The pipeline has been assembled to address the data coming from ESO optical/infrared imagers. It is also used as the image processing engine of the end-to-end EIS survey system mentioned above. DISCLAIMER Even though best effort has been made to ensure the accuracy of the package and manual, both still remain a work in progress. Therefore, we encourage potential users to send their comments, suggestions and verification results via email to: eis-help@eso.org Current users of the system include: S. Arnouts (Marseille); C. Benoist (Nice Observatory); C. Mullis (ESO Fellow/AVO); S. Zaggia (Trieste Observatory). The system is also being evaluated by the DFS/DMD of ESO. Users of this preliminary version accept the following conditions: - the code in its present form is the property of ESO and as such follows the regulations described in the ESO Software Policy document of March 7, 2003 (GEN-POL-ESO-00000-3011 V1.0) - the transfer of the code will be made using the survey system framework so as to register the distribution for future reference - all errors/bugs/suggestions should be reported via email to: eis-help@eso.org - all reports should explictly refer to software version in use and date of release (e.g. Version: 1.0.1, date of release: 6 February 2004). - the code may change without prior warning. It is the responsibility of the user to check for updates - the EIS team cannot guarantee immediate action on problems reported by external users. The PSG reserves the right to establish priorities - the EIS team takes no responsibility for the scientific use of the package in its current form - the EIS team takes no responsibility for any changes made to the source code by the users - communication among users is strongly urged to exchange experiences COPYRIGHTS The EIS/MVM is a copyright-protected software product of the European Southern Observatory for image processing. It is available under the GNU General Public License as published by the Free Software Foundation. Users of this package are kindly requested to acknowledge the European Southern Observatory and the EIS Project for the use of the software in any publication or presentation of results prepared using this package. They are also requested to reference Vandame et al. (2003, in preparation). REFERENCES da Costa, L., 2003, ``Making Ground-based optical/infrared imaging surveys VO-compatible'', "Toward an International Virtual Observatory - Scientific Motivation, Roadmap for Development and Current Status, ESO Conference Series, in press da Costa, L., 2003, Astronomical Data Analysis Software & Systems XIII October 12-15, 2003 Vandame, B. 2002, "New algorithms and technologies for the un-supervised reduction of Optical/IR images" Proceedings of the SPIE, Volume 4847, pp. 123-134 (2002)