DRAFT procedure for creating ALMA MSs upon user request

Purpose

The ALMA project presently only stores raw data in its archive. Users are given software to calibrate the data after download. It was decided by ALMA management that a new service be offered by ESO where users can request the creation and staging of calibrated data for download.

Procedure Summary

The user requests the calibrated data via a helpdesk ticket. The helpdesk ticket should contain the MOUS UID(s) (up to 10) of the data to calibrate. The ticket is received by designated EU ARC personel who hands on the MOUS UID together with the ALMA user id is by email to AOG.

AOG verifies that the user is entitled to access the data on the day of the request (i.e. the day when AOG receives it). If entitlement is verified, AOG inserts the MOUS UID into a standard command line which is run by AOG on almasciencedev.hq.eso.org as user "arcdistribution".

The command ("stagecalms" developed and tested by D. Petry) connects to the EU ARC cluster and triggers the creation of the calibrated MS. It tars up the completed MS and moves it into a new directory on almasciencedev.hq.eso.org. When the command line completes, it prints to the terminal the newly created URL where the data has been staged.

If there is more than one MOUS to calibrate, AOG can run up to two staging commands in parallel. If there are more than two MOUSs, the commands need to be issued sequentially.

AOG takes the resulting URL(s) and communicates them to the user via a standard email using the user's email on record with the EU ARC personel in CC who handed on the request.

The EU ARC personel then closes the helpdesk ticket.

Detailed Procedure

Create email to AOG after receipt of a request via helpdesk

For each requested MOUS, the helpdesk personel should send one email to AOG.

The email should have the following format:

To: aog@eso.org
CC: dpetry@eso.org
Subject: "ALMA: REQUEST FOR CALIBRATED DATA MOUS uid___A001_X133d_X1244"

Body: 
Dear AOG,
please create the calibrated MS for MOUS uid___A001_X133d_X1244
project <project code>
for user <ALMA User ID>
helpdesk ticket <helpdesk ticket number>

Thanks!

Verification of entitlement and availability

After receiving the request email, AOG uses the contained user ID and the MOUS UID to determine whether the user is entitled at that moment to obtain this data. F. Stoehr has provided a draft query:

select asa_science.project_code,b.firstname PI_FIRSTNAME,b.lastname
PI_LASTNAME, b.email PI_EMAIL, b.account_id PI_ACCOUNTID, b.preferredarc
PI_ARC, a.firstname DELEGEE_FIRSTNAME,a.lastname
DELEGEE_LASTNAME,a.email DELEGEE_EMAIL, a.account_id DELEGEE_ACCOUNTID,
a.preferredarc DELEGEE_ARC from alma.dp_delegation join alma.account a
on delegee_rh_id=a.REQUEST_HANDLER_ID join alma.account b on
PI_RH_ID=b.REQUEST_HANDLER_ID join alma.asa_science on
asa_science.project_code = dp_delegation.project_code where
member_ouss_id = 'uid://A001/X134b/X41' group by
asa_science.project_code,b.firstname ,b.lastname , b.email,
b.account_id, b.preferredarc, a.firstname ,a.lastname,a.email,
a.account_id, a.preferredarc

where the MOUS ID uid://A001/X134b/X41 of course has to be replaced with the MOUS ID that one wants to check

If the user is entitled, AOG also checks if the MOUS is in state delivered.

Reply to helpdesk personel in case the user is not entitled

If AOG determines that the user is not entitled at the moment to access the data, AOG replies to the request email and writes the text

NOT ENTITLED

above the quoted original email text.

This concludes AOG's involvement.

The helpdesk personel replies via helpdesk to the user using the following text:

You presently do not have access permission for the requested data.
Please wait until the data becomes public.

Reply to helpdesk personel in case the user or the MOUS does not exist

If AOG determines that the data is faulty, i.e. the MOUS UID or the user id is invalid, AOG replies to the request email and writes the text

INVALID USER ID

or

INVALID MOUS UID

above the quoted original email text.

The helpdesk personel can then communcate with the user and try to fix the information.

Reply to helpdesk personel in case the MOUS is not in state delivered

If AOG determines that the MOUS UID is not in state delivered, AOG replies to the request email and writes the text

MOUS NOT DELIVERED

above the quoted original email text.

The helpdesk personel can then investigate further the status of the MOUS and tell the user what can be done.

Creation and staging of the MS(s)

With the information from the request email, AOG proceeds to run the MS creation on almasciencedev.hq.eso.org:

AOG logs in as arcdistribution@almasciencedev.hq.eso.org and types

stagecalms.sh PROJCODE MOUSUID HELPDESKTICKETNUM

This will trigger jobs on the EU ARC cluster.

Up to two stagecalms instances can be run at the same time.

The typical terminal output of the command looks like this

The staging directory will be accessible at http://almascience.eso.org/arcdistribution/preview/cdb386950af8ee148b3a9bc633942c6a
Looking for cluster node with low load ...
DATAMACHINE chosen: arcp9
Working as arcproc@arcp9.hq.eso.org ...
(re-)creating directory /opsw/work/pipeproc/autopipeline/tmp/WORK11018
Now generating the calibrated MS(s) ...
Checking for pre-existing MSs for the following EBs ...
uid___A002_X7e4988_X1f2
MS for uid___A002_X7e4988_X1f2 was not found among the already staged MSs. Will not search further ...
None or not all of the needed MSs found: Will regenerate all the MSs ...
Copying over the generated MSs ...
receiving incremental file list
uid___A002_X7e4988_X1f2.ms.split.cal.tar

sent 42 bytes  received 15449500342 bytes  95073848.52 bytes/sec
total size is 15445729280  speedup is 1.00
Deleting redundant logs ...
Deleting temporary work directory ...

Staging of calibrated MS(s) for 2012.1.00357.S uid___A002_X788a57_X29 helpdesk ticket #15270 completed
at http://almascience.eso.org/arcdistribution/preview/cdb386950af8ee148b3a9bc633942c6a
Data will be deleted 28 days from Wed Jul 17 17:09:18 UTC 2019unless you put a .donotremove into the directory.

There can be the additional information

Old project, using RHEL5 build of CASA.

which indicates that the required CASA version is so old that it needs to be run on a special RHEL5 machine. This message is just for information.

Re-use of existing calibrated MSs on almasciencedev

stagecalms has the special feature that it will minimise the computing effort for the data calibration by first searching for existing copies of the requested calibrated MS on almasciencedev.

If a copy of the MS already exists (because a different user requested it less than 28 days before the present request), then, instead of recreating the MS again, stagecalms will simply create a hard link to the existing MS in the new download directory.

In case this happens, stagecalms completes very quickly and there are messages like

Found pre-existing /home/arcdistribution/arcdistribution/preview/8b6db8a851c558b1669e7a0d987e6f87/uid___A002_Xd15514_X8bfa.ms.split.cal.tar .
   Will try to use this instead of recreating it (making hard link) ...

on the terminal.

Otherwise, there is a message

MS for uid___A002_X9aa6ef_X15a1 was not found among the already staged MSs. Will not search further ...
None or not all of the needed MSs found: Will regenerate all the MSs ...

The messages are just for information. AOG does not have to take special action either way.

What to do in case stagecalms completes with an error

In case there is an error, stagecalms will produce a terminal output similar to

Failed. Please investigate  /var/www/html/arcdistribution/preview/cdb386950af8ee148b3a9bc633942c6a and possibly remove incomplete data belonging to MOUS uid://A001/X11f/X9c
The working directory /opsw/work/pipeproc/autopipeline/tmp/WORK23486 on the EU ARC cluster was not removed.

The entire terminal output including the command line which was entered by AOG should be copied and pasted into an email to dpetry@eso.org for investigation.

dpetry will then give further instructions.

Email to the PI after successful completion of stagecalms

Once stagecalms has completed (which can take hours), AOG should create an email and send it to the user with the EU ARC helpdesk personel and dpetry@eso.org in CC.

The format of the email should be the following:

To: <alma user email address obtained via ALMA user id>
CC: <original sender of the request email>, dpetry@eso.org
Subject: "Re: ALMA: REQUEST FOR CALIBRATED DATA MOUS uid___A001_X133d_X1244"
Body: 
Your calibrated data is ready for download from
http://almascience.eso.org/arcdistribution/preview/cdb386950af8ee148b3a9bc633942c6a

It will be deleted 28 days from now.

Kind regards,

The ESO ALMA Team

where the download link is copied and pasted from the terminal output of stagecalms in the previous step.

In other words, AOG just does reply-all on the original request email and adds the email of the ALMA user as prime "to" address.

This concludes AOG's work for the given MOUS.

The staged data will be automatically deleted after 28 days by a cronjob on almasciencedev.

Helpdesk personel closes helpdesk ticket

Once the helpdesk person receives the email with the download link from AOG, he./she copies and pastes the email into a last comment on the ticket and closes the ticket.

-- Dirk Petry - 2019-08-05

Comments