Difference: ALMACalMSCreation (9 vs. 10)

Revision 102019-10-01 - DirkPetry

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Procedure for creating ALMA MSs upon user request

Changed:
<
<		(version 2019-09-20)
>
>		(version 2019-10-01)
 
Line: 35 to 35
 If entitlement is verified, AOG inserts the MOUS UID into a standard command line which is run by AOG on almasciencedev.hq.eso.org as user "arcdistribution".
Changed:
<
<		The command ("stagecalms" developed and tested by D. Petry) connects to the EU ARC cluster
>
>		The command ("stagecalms" developed and maintained by D. Petry) connects to the EU ARC cluster
 and triggers the creation of the calibrated MS. It tars up the completed MS and moves it into a new directory on almasciencedev.hq.eso.org. When the command line completes, it prints to the terminal the newly created URL where
Line: 64 to 64
 

Create email to AOG after receipt of a request via helpdesk

Changed:
<
<		For each requested MOUS, the assigned EU ARC personnel should send one email to AOG.
>
>		For each requested MOUS, the assigned EU ARC personnel should send one email to AOG via the special script "triggerCalMSCreation".
  The email should have the following format:
Line: 97 to 97
 triggerCalMSCreation.sh 2017.1.00727.S "Judit Szulagyi" judits@phys.ethz.ch 16140 uid://A001/X1296/X675
Changed:
<
<		This script (developed by D. Petry) sends the above email to three addresses: the sending EU ARC person (cluster user), to AOG, and dpetry,
>
>		This script (developed and maintained by D. Petry) sends the above email to three addresses: the sending EU ARC person (cluster user), to AOG, and dpetry,
 and places a textfile with an equivalent of the email into a special directory on the cluster:
Line: 144 to 144
 helpdesk ticket 16140
Changed:
<
<		AOG will install a cronjob on an AOG machine to look for new triggers once per hour.
>
>		AOG is running a cronjob on an AOG machine to look for new triggers once per hour.
 

Prevention of duplicate triggers

Line: 158 to 158
  After receiving the request email, AOG uses the contained user ID and the MOUS UID to determine whether the user is entitled at that moment to obtain this data.
Changed:
<
<		F. Stoehr has provided a draft query: 
select asa_science.project_code,b.firstname PI_FIRSTNAME,b.lastname
PI_LASTNAME, b.email PI_EMAIL, b.account_id PI_ACCOUNTID, b.preferredarc
PI_ARC, a.firstname DELEGEE_FIRSTNAME,a.lastname
DELEGEE_LASTNAME,a.email DELEGEE_EMAIL, a.account_id DELEGEE_ACCOUNTID,
a.preferredarc DELEGEE_ARC from alma.dp_delegation join alma.account a
on delegee_rh_id=a.REQUEST_HANDLER_ID join alma.account b on
PI_RH_ID=b.REQUEST_HANDLER_ID join alma.asa_science on
asa_science.project_code = dp_delegation.project_code where
member_ouss_id = 'uid://A001/X134b/X41' group by
asa_science.project_code,b.firstname ,b.lastname , b.email,
b.account_id, b.preferredarc, a.firstname ,a.lastname,a.email,
a.account_id, a.preferredarc
where the MOUS ID uid://A001/X134b/X41 of course has to be replaced with the MOUS ID that one wants to check
>
>		The database queries to achieve this were developed together with F. Stoehr. AOG is maintaining the queries.
  If the user is entitled (i.e. is the PI or a delegee as identified by the user email), AOG also checks if the MOUS is in state delivered and if the MOUS is not in state QA3InProgress.
Added:
>
>		Only if these conditions are met, AOG goes ahead and starts the creation of the calibrated MS.
 

Reply to EU ARC personnel (helpdesk contact) in case the user is not entitled

If AOG determines that the user is not entitled at the moment to access the data,

Line: 248 to 236
 The typical terminal output of the command looks like this
Changed:
<
<		The staging directory will be accessible at http://almascience.eso.org/arcdistribution/preview/cdb386950af8ee148b3a9bc633942c6a
>
>		The staging directory will be accessible at http://almascience.eso.org/arcdistribution/preview/8672af71dac32e0ad9703dae3438fcac
 Looking for cluster node with low load ...
Changed:
<
<		DATAMACHINE chosen: arcp9 Working as arcproc@arcp9.hq.eso.org ... (re-)creating directory /opsw/work/pipeproc/autopipeline/tmp/WORK11018
>
>		DATAMACHINE chosen: arcp14 Working as arcproc@arcp14.hq.eso.org ... (re-)creating directory /opsw/work/pipeproc/autopipeline/tmp/WORK23067
 Now generating the calibrated MS(s) ... Checking for pre-existing MSs for the following EBs ...
Changed:
<
<		uid___A002_X7e4988_X1f2 MS for uid___A002_X7e4988_X1f2 was not found among the already staged MSs. Will not search further ...
>
>		uid___A002_Xd33a12_X15be MS for uid___A002_Xd33a12_X15be was not found among the already staged MSs. Will not search further ...
 None or not all of the needed MSs found: Will regenerate all the MSs ...
Added:
>
>		Generated the job script. Now submitting and waiting for completion ... Completed. Determining the list of generated MSs ...
 Copying over the generated MSs ... receiving incremental file list
Changed:
<
<		uid___A002_X7e4988_X1f2.ms.split.cal.tar
>
>		uid___A002_Xd33a12_X15be.ms.split.cal.tar
 
Changed:
<
<		sent 42 bytes received 15449500342 bytes 95073848.52 bytes/sec total size is 15445729280 speedup is 1.00
>
>		sent 42 bytes received 5765728719 bytes 64421550.40 bytes/sec total size is 5764321280 speedup is 1.00
 Deleting redundant logs ... Deleting temporary work directory ...
Changed:
<
<		Staging of calibrated MS(s) for 2012.1.00357.S uid___A002_X788a57_X29 helpdesk ticket #15270 completed at http://almascience.eso.org/arcdistribution/preview/cdb386950af8ee148b3a9bc633942c6a Data will be deleted 28 days from Wed Jul 17 17:09:18 UTC 2019unless you put a .donotremove into the directory.
>
>		Staging of calibrated MS(s) for 2018.1.01406.S uid___A001_X133d_X3cf3 helpdesk ticket #16267 completed at http://almascience.eso.org/arcdistribution/preview/8672af71dac32e0ad9703dae3438fcac Data will be deleted 28 days from Fri Sep 27 14:04:04 UTC 2019 unless you put a .donotremove into the directory.
 

There can be the additional information

Line: 355 to 345
 Once the helpdesk person receives the email with the download link from AOG, he./she copies and pastes the email into a last comment on the ticket and closes the ticket.
Changed:
<
<		-- Dirk Petry - 2019-08-05
>
>		-- Dirk Petry - 2019-10-01
 

Comments

View topic | History: r13 < r12 < r11 < r10 | More topic actions...
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding ARC TWiki? Send feedback