The Joint ALMA Observatory (JAO) has carried out a detailed investigation into the cause of the performance degradation of the servers at the JAO that led to the issues experienced by some ALMA users that were described in a previous announcement. It was found that a combination of factors led to server performance degradation, consisting of unusually large system log files generated in the JAO network storage, which affected various proposal submission services, as well as  added load due to a vulnerability monitoring service.  Additionally, some users experienced login difficulties due to an issue with the error handling in the submission service. The effects of these issues were exacerbated during the approximately three hours prior to the proposal deadline when the number of submissions is always the greatest. 

ALMA sincerely regrets the distress these issues caused to users trying to submit their proposals and is actively taking steps to address the identified technical issues and ensure that this situation does not occur again in future cycles.  

As described in the previous announcement, all cases in which the proposal was submitted  with incorrect observing time estimates were corrected by the Observatory before proposals were sent to review. Additionally, proposal reviewers were instructed to disregard any inconsistencies between the observing time listed on the proposal cover sheet and  that written in the Scientific or Technical Justification. ALMA understands that affected users may wish to see the proposals sent to reviewers with corrected observing times. Users can access the updated proposal pdfs in SnooPI via the “Download Proposal [pdf]” button at the top-right of the proposal window (reached by clicking on the project code in the list of projects).  

 If you have any questions, or have comments or concerns, please contact your local ALMA Regional Centre via the ALMA Helpdesk at https://help.almascience.org/.