Once per year, now in the northern autumn, the User Support Department of ESO launches a Paranal Service Mode User Satisfaction Survey campaign. While this survey is by default anonymous, respondents are afforded the opportunity to identify themselves and to provide their e-mail address (see below).

This report details the findings of the September/October 2023 survey campaign (hereafter referred to simply by October 2023), while previous such reports are found here.

We view these reports as an important way to

• close the loop with the ESO Community,
• gather information on issues that need to be addressed or reinforced,
• thank all respondents, and
• demonstrate clearly that such feedback is important to us! 

To this end, here we provide a summary of the responses received and trends in these responses over the last years, predominantly in the form of graphs. It should also be stressed that for those few cases where respondents did identify themselves and did make specific free-text comments we have contacted them by e-mail to address their particular comments.

Methodology and General Results

The ESO Service Mode Questionnaire is always available on-line for users to fill in but the usual rate of return is less than 2 per month. However, experience shows that a targeted campaign focused on a single (in this case Phase 2 related) aspect results in many more survey completions.

In September 2023, we asked Principal Investigators (PIs) of Service Mode runs scheduled for Paranal in Periods 111 and/or 112 (plus their then-active Phase 2 delegates) to complete the survey by a fixed deadline. We thus solicited a response from 466 PIs and their then-active Phase 2 delegates (426 individuals). Because of overlap this amounts to a total of 764 individuals, who were contacted via e-mail. A deadline was set for three weeks from the date of contact, and one reminder e-mail was sent before the deadline.

A total of 146 responses were received by the deadline (some 32 of which were not fully complete), representing a 19.1% return rate.

Interactive Figure Features

The figures below are all interactive.  By this we mean:

• Puttling the cursor over the plot will display the data values on the screen.
• Clicking on the menu icon in the upper right (the three short parallel horizontal lines) will open a menu of print/download options.
• For those figures with legends to the right of the plot clicking on any entry in the legend will toggle display of the corresponding data within the graph.

As a start in detailing the results from the survey, in the figure below we show the number of responses we received per instrument.  The fact that this year's survey covered 15 instruments (as opposed to 14 last year), coupled with the overall lower response rate, resulted in a decrease in the average number of responses per instrument (11.3 in contrast to last year's value of 19.8).

In the following two stacked histogram plots we present a general overview of user satisfaction (in percentage of responses) with two general items:

• the Phase 2 web documentation and
• the overall support provided by the User Support Department.

The plots, designed to show the trend in user satisfaction expressed in survey results since September 2015, clearly show the consistently high satisfaction with these services offered by the User Support Department.

p2 and Other Observation Preparation Tools

As in the past, we also asked about both the p2 tool and other, instrument-specific, observation preparation tools. For comparative purposes we include still the last P2PP results (through September 2018) for a direct side-by-side comparison of P2PP and p2. In general the satisfaction levels with these tools are somewhat poorer than with the above mentioned services provided by the User Support Department.  p2 is still overall well accepted, though we note that both ease of use and documentation satisfaction are the lowest values shown in these figures, even lower than for its predecessor, P2PP.  Thus there remains room for improvement on both items. The user satisfaction with functions provised by the p2 application remains consistent with the previous year's value.

ESO has released a Phase 2 Application Programming Interface (API) that can be used to create, modify, or delete observation blocks (OBs), containers and an accompanying ReadMe file that define an observing run.  We asked respondents whether or not they had made use of this powerful facility.  The way they replied is shown below. In that plot we see that fractional use of the p2API is apparently unchanged from last year, but the reader should be alerted to the small number statistics here.

Since, with the exception of ObsPrep, the number of responses per observing preparation tool other than p2 is rather limited (see the table below), any presentation of individual-tool responses on documentation, ease of use, or functionality would suffer from small number statistics. Thus, in the three figures below answers for all tools, except ObsPrep, are combined. The ObsPrep figures can be found further down the page.  With the possible exception of the ease of use question, which does show steady improvement since 2020, when one considers the other tools as an emsemble we do find scope for (continued) improvement.

As the usage of the ObsPrep tool outweighs all the other tools combined (see above) we have separated out responses to our ObsPrep questions for display below.  Starting with usage statistics in the upper left plot we see a suggestion that usage is (slowly) increasing.  At least part of that increase can be potentially attributed to the addition of ERIS since last year's survey.  The community does seem to appreciating ObsPrep more and more.

The ObsPrep documentation satisfaction level (top right plot below) shows that  our efforts to turn around the previous decline in very satisfied users seems to have paid off.

ObsPrep ease of use and functionality provided (bottom two plots) both show very strong user satisfaction levels, though both show (potrenially worrisome) decreases in the percentages of very satisfied users.

Related to the above tools is, of course, the suite of Exposure Time Calculators.  Thus, we asked survey participants the question, “How satisfied are you with the ETCs you have used?” The responses are shown below. There are consistently few respondents that express dissatisfaction with the ETC (and we do try to follow up comments when given). The apparently increasing percentage of "No opinion" answers we interpret as that fraction of the respondents that did not use any ETC.

Again we asked users to tell us how satisfied they are with the p2fc (finding chart generator) app within p2.  We asked how they liked the tool in terms of documentation and its usefulness (as compared to any other alternatives for producing finding charts), with the plots below displaying the results.  The user satisfaction with the documentation appears to be essentially the same as in last year's survey.

And lastly we asked survey participants, "Which operating system(s) do you use for ESO tools (e.g. for proposal/observation preparation, data reduction), excluding any browser-based tools?" This information is obviously not used as an indicator of satisfaction with the tools or support, but rather is gathered to ensure that we have an understanding about the technical requirements that users may have for the tools. We strive to develop tools that are OS independent, but a few of the legacy observation preparation tools still require local installation. Furthermore, data reduction pipelines also need to be packaged and tested on specific OSs.

The breakdown of responses is shown in the figures below.  Here we see Linux overtaking Mac OS X on this occasion, with a similarly small percentage of Windows usage.

Within the Linux usage it appears that Ubuntu is still dominant.

Notes:

¹The total time allocated in Service Mode for Periods 111 and 112 was 10686.4 hours, while the corresponding number for Visitor Mode was 1467.1 hours.  Thus, the October 2023 survey targetted PIs (and their then-active delegates) representing 87.9% of the total Paranal time allocation, including all public surveys. This is a quite similar fraction as for the previous year’s survey campaign (86%).