Definition, Purpose, Coverage and Completeness of telbib

Definition
The ESO Telescope Bibliography (telbib) is a database of refereed papers published by the ESO users community. All papers use partly or exclusively data from ESO facilities. Statistics derived from the ESO Telescope Bibliography include only papers based on data from telescopes and instruments for which observing time was recommended by ESO.

telbib is compiled by scanning the major astronomical journals for scientific papers that contain any of the ESO-defined keywords (e.g., telescope and instrument names). All papers included in the database have been inspected visually to ensure that they directly use ESO observational data.

For further details please refer to the telbib Workflow section and Fig. 1 below.

Purpose
The ESO Telescope Bibliography is predominantly used by project and instrument scientists, ESO Management (and through them, the ESO Council) as well as the ESO users community at large. The main purposes of the database are to help with the following aspects:

• interconnect resources -- from proposals to papers and from papers to data in the archive
• measure ESO's scientific output (productivity and impact)
• evaluate the performance of telescopes and instruments
• put ESO in context with other observatories
• define guidelines for future facilities

It is important to note that bibliometrics are only one aspect of these scopes; other evaluation processes have to be used in addition to achieve a more complete picture.

Coverage
telbib contains records from publication year 1996 onwards. New records are added approx. 3-6 weeks after they appear with their final bibcode on the NASA ADS Abstract Service. As of October 2018, the following journals are routinely screened: A&A, A&ARv, AJ, AN, ApJ, ApJS, ARA&A, EM&P, Icarus, MNRAS, Nature, NewA, NewAR, PASJ, PASP, P&SS, Science. Papers from other journals are added iwhen we retrieve them.

Completeness
We make extensive efforts in order to identify all refereed papers that use ESO data including

• semi-automated screening of journals for ESO-defined keywords
• visual inspection of each paper that contains at least one of the keywords
• identification and cross-checking of all instruments and program IDs used for the research
• querying of the ESO Observing Schedule and ESO Science Archive to obtain program IDs not mentioned in the paper
• correspondence with authors and instrument scientists in case of doubt

Because of this meticulous follow-up work, telbib is considered to be essentially complete. If you are aware of a paper that is missing from the telbib database, please contact us at esodata@eso.org.
Statistics for a given year will not be complete until approx. March of the following year.

Paper classification

Papers pertaining to the ESO Telescope Bibliography use partly or exclusively data from ESO facilities. These can be observations taken by the authors or data obtained from the ESO Science Archive or other sources, regardless of whether or not the observations have been published before.

However, papers that merely quote results or measurements from articles that use ESO data, are excluded. Likewise, papers that describe instrumentation or software, simply mention ongoing projects (e.g., surveys or Large Programmes), suggest future observations with ESO facilities, develop models or run simulations, using data merely as examples or for training (e.g., for Machine Learning), are not included in telbib. Also excluded are papers which show ESO images as a visual reference rather than using them to achieve scientific results.

The ESO librarians communicate extensively with authors as well as ESO instrument scientists and archive specialists to determine if, and which, observations were used in publications. The final decision about inclusion or exclusion of a given paper lies with the ESO Director for Science.

telbib Workflow

The ESO Library has developed two tools in order to maintain the Telescope Bibliography: (1) FUSE is a full-text search tool that semi-automatically scans defined sets of journal articles for organizational keyword sets, while providing highlighted results in context; (2) the Telescope Bibliography (telbib) is used to classify ESO-related papers, store additional metadata, and generate statistics and reports. Both tools rely heavily on the NASA ADS Abstract Service for bibliographic metadata.

FUSE and telbib form part of a workflow that links published literature with data located in the ESO archive. The result is an information system that answers predominantly two questions:

• Which ESO facilities generated the data used in the scientific literature?
• Which publications used data provided by specific ESO facilities?

The essential components of FUSE and telbib are described below. The selection criteria for inclusion or exclusion of papers into telbib have been defined and are applied in compliance with other observatory bibliographies, for instance those of STScI and NRAO.

1. Access to literature
   

   In order for FUSE to work, it is necessary to have access to the electronic versions of all scientific journals that shall be monitored. FUSE provides search methods for these journals that allow retrieval of the full-texts of articles (typically in PDF format) from the publishers’ websites. The PDFs are not stored permanently on ESO servers.

2. FUSE full text search tool
   

   FUSE is a PHP/MySQL tool created by the ESO Library. It converts PDFs into text files and scans them for user-defined keyword sets. If any keywords are detected in the text files, they are highlighted and shown in context on the results page. After inspection, the text files are deleted.

3. telbib back-end: bibliographic details and further metatdata
   

   After FUSE has identified possible candidates for the ESO Telescope Bibliography, these papers are inspected visually in detail to make sure that they do use ESO data. False hits (e.g., when VLT is the abbreviation for Very-Low Temperature rather than Very Large Telescope) are eliminated immediately. Records that shall be added to telbib are imported into the database through the librarians’ interface (telbib back-end, implemented with PHP/Sybase). Bibliographic information (authors, title, publication details) along with further metadata like current number of citations, author-assigned keywords, and author affiliations are imported from the NASA ADS Abstract Service.

4. telbib tags and program IDs
   

   telbib only includes refereed papers that use partly or exclusively data from ESO facilities. Statistics derived from the ESO Telescope Bibliography include only papers based on data from telescopes and instruments for which observing time was recommended by the ESO. Instrumentation and other engineering papers that do not actually use data are excluded, as well as papers that use images as overlays without further analysis.

   Likewise, we take the meaning of "refereed" publications very seriously and exclude for instance conference proceedings even if they were published in otherwise refereed journals (e.g., a special journal issue presenting only proceedings of a symposium).

   The ESO librarians extensively tag and annotate each telbib record. Tags include standardized descriptions of telescopes, instruments, surveys, and other information.

   Most importantly, program IDs and Data Collection DOIs that provided data are assigned to the telbib record of the paper, including the program's observing mode (visitor or service mode) and type (normal program, Target of Opportunity (TOO) program, Large Program, etc.). For APEX and ALMA papers, we also take note of the project partner (ESO, Max-Planck, Onsala, Chile for APEX and Europe, North America, East Asia, and Chile in the case of ALMA). This allows us to retrieve from the telbib database papers based on data resulting from observing time from any APEX or ALMA partner as well as those papers that use ESO/Europe observing time. telbib statistics only include papers that used observations obtained during ESO/Europe time.

   The ESO data citation policy (Publications with ESO Data) asks authors to mention ESO program IDs in the footnote or an acknowledgement of their papers. All program IDs listed by authors are verified to eliminate possible typos and other mistakes. In case the authors do not mention the exact program IDs (or list only the most obvious ones while omitting IDs for supporting data which were also used), we seek to determine them by querying the ESO observing schedule and the ESO archive for observing dates, instruments, and other information regarding the observations provided by the authors.

   For each paper, we note whether these are PI (Principle Investigator) observations (i.e., at least one of the authors has also been PI/CoI on the observing proposal), archival data (i.e., there is no overlap between authors and observer team), or a mix of both (when PI as well as archival data were used). Papers that use so-called data products are tagged in such a way that credit is also given to the work and infrastructure provided by the ESO Science Archive.

   In case we cannot determine the exact program or, even more importantly, whether or not data were used at all, we contact the authors and ask for clarification. Typically, authors are very helpful and respond fast and precisely. However, sometimes we do not receive an answer. In such cases, we usually contact for instance an ESO scientist who was involved in the development and construction of the instrument because instrument scientists are often familiar with observations that were carried out at their facility. If it still cannot be decided whether or not ESO data were used, we exclude the paper from telbib.

   The ultimate decision regarding telbib policies and inclusion or exclusion of papers rests with the ESO Director for Science in cooperation with the librarians.

5. Data archive

   The program IDs assigned to telbib records provide links to the corresponding data in the ESO Science Archive. In this way, readers of scientific papers who are interested in the data used in the publication can easily find the observing programs. All program IDs in telbib will have links to the raw data in the archive. If data products exist, the link will be displayed too, regardless of whether or not raw data or data products were used in the paper. The data can be requested after the proprietary period (typically one year, unless a different policy applies).

6. telbib front-end

   telbib can be searched through the public user interface (https://telbib.eso.org). As described above, the records list all programs that generated the data used in the paper and provide links to the ESO archive (raw data as well as dtata products, if available) through the program IDs. In return, program IDs in the ESO Science Archive provide links back to telbib, listing all scientific papers that use specific observing programs. More information about the telbib front-end can be found in the telbib Help page.

Facility and Program Tags

Chajnantor
All papers using data from Chajnantor facilities (APEX and/or ALMA) are tagged with telescopes and program IDs. In addition, for APEX-based papers instruments are recorded. In contrast, ALMA-based papers are not tagged with the specific receiver bands.

All APEX and ALMA science papers are retrievable through the public telbib interface, regardless of the project partner (ESO, Max-Planck-Society, Onsala, Chile for APEX; Europe, North America, East Asia, Chile for ALMA) that provided the data. Users can limit their searches to ESO/European data papers by choosing the option "Only ESO data papers (excluding non-ESO ALMA / non-ESO APEX papers)".  ESO's statistics and reports, e.g., the 'Basic ESO Statisics' document, only take those papers into account that use data from ESO/European observing time.

Paranal
All papers using data from Paranal facilities (i.e., VLT, VLTI, VISTA, and/or VST) are tagged with telescopes, instruments, and program IDs.

La Silla
Papers using data from La Silla telescopes (selected instruments mounted at NTT, 3.6 m telescope, 2.2 m telescope, and/or 1.5 m telescope) are tagged as follows:
La Silla telescopes: from 1996 onwards
La Silla instruments: from 2000 onwards
La Silla program IDs: from 2000 onwards
Some instruments (e.g., FEROS@1.5 m telescope) may have been operated under special agreements for which no program IDs exist. For information on papers using data from other (possibly by now decommissioned) La Silla telescopes, please contact the librarians.

If you have any questions, need specific reports or have suggestions regarding telbib, contact the ESO librarians at library@eso.org.