(r49) ESOCASAPipelineMeeting2019 < External

External Web>ESOCASAPipelineMeeting2019 (revision 49) (raw view)~~Raw edit~~ | ~~EditAttach~~

---+ ESO CASA and Pipeline Meeting - 15~17 May 2019

<br />%TOC%

---++ 1. Travel to ESO

   * [[https://www.eso.org/public/about-eso/travel/garching/][Getting to ESO and hotels in Garching]]
   * For a list of hotels in Munich please see [[http://www.muenchen.de/Tourismus/Accommodation_Hotels/7604/index.html][Munich hotels]]. If you prefer to stay in Munich, we suggest to choose a hotel close to the U6 metro line.
   * Visiting the ESO Supernova: [[https://supernova.eso.org/programme/calendar][programme]]

---++ 2. Participants
   * External:
      * Kana Sugimoto (NRAO - NM)
      * Ryan Raba (NRAO - CV),
      * Dirk Muders (MPIFR - Bonn),
      * Stewart Williams (UKRI-STFC - Edinburgh)
   * ESO:
      * Vinod Arumugam
      * Andy Biggs,
      * Sandra Castro,
      * Alex Chalevin
      * Maurizio Chavan,
      * Maria Diaz,
      * Enrique Garcia,
      * Justo Gonzalez,
      * Fabrizia Guglielmetti,
      * Liz Humphreys,
      * Alisdair Manning,
      * Luke Maud,
      * Stewart Mclay,
      * Federico Montesino,
      * Tony Mroczkowski,
      * Dirk Petry,
      * Suzanna Randall,
      * Erich Schmid
      * Felix Stoehr,
      * Martin Zwaan
   * Remote participants via VC:
      * John Hibbard (NRAO - CV)
      * Remy Indebetouw (NRAO - CV)

---++ 3. Video Connection

To connect to a meeting please use one of the following connection methods

Video Conference, use the following address meet.room1@eso.org

Telephone +49 893200 69441

*With Web RTC (Google Chrome, Safari, Microsoft Edge or Firefox)* https://join.eso.org/invited.sf?secret=bNrpMvJSDcN8MImw_RU4xw&id=320069441

---++ 4. Dinner in Munich

Date:
<div style="padding-left: 30px;"> *16th May at 19:30* </div> <literal>Place:</literal> <div style="padding-left: 30px;"> *Paulaner Bräuhaus* </div> <div style="padding-left: 30px;">Kapuzinerplatz 5, 80337 München</div> <div style="padding-left: 30px;">Telefon +49(0)89 / 544 611-0</div> <div style="padding-left: 30px;">https://www.paulaner-brauhaus.de/</div> <div style="padding-left: 30px;">Getting there by public transport from Garching: see this [[%ATTACHURLPATH%/PaulanerBraeuhausMap.pdf][map]].</div>

---++ 5. Agenda

*- Meeting room: Pavo, A.2.01 -*
---++++ Wednesday, 15 May
   * 9:30 - Welcome and Introductions (Sandra/Luke)
      * - [[https://www.eso.org/sci/meetings/2018/tcl2018/code.html][ESO Code of Conduct]]
   * 9:45 - Pipeline status today (Kana)
   * 10:15 - [[%ATTACHURLPATH%/Pipeline_Overview.pdf][Overview of the Pipeline and how it works]] (Stewart)
      * - Pipeline: weblogs, content, how, where, and what connections. Tips and Tricks (Stewart)
   * 11:15 - <span class="WYSIWYG_COLOR" style="color: royalblue;">Coffee break</span>
   * 11:30 - [[%ATTACHURLPATH%/Interferometry-Imaging-Status-EU-ARC-May-2019.pdf][Pipeline: imaging status and inside view]] (Dirk M.)
   * 12:15 - Lunch
   * 13:30 - [[%ATTACHURLPATH%/CASA_Status_201905.pdf][Pipeline development process and the Integration and Performance team]] (Ryan/Kana/Sandra)
   * 14:00 - [[%ATTACHURLPATH%/eso_casa_pl_201905_parallelization_performance.pdf][CASA and Pipeline: status of parallelisation and performance]] (Federico)
      * - Pipeline failures; MPI timeouts and what they mean (Federico)
      * - CASA memory usage and monitoring (Enrique)
   * 15:15 - <span class="WYSIWYG_COLOR" style="color: royalblue;">Coffee break</span>
   * 15:30 - Discussion topic: Group-level Processing, how to make best progress (Felix/all)
   * 16:00 - Information topic: TelCal and testing for the upcoming Correlator Upgrade (Justo)
---++++ Thursday, 16 May
   * 9:30 - [[%ATTACHURLPATH%/CASA_Status_and_FowardPlan201905.pdf][CASA status today]] (Ryan)
      * - CASA forward plan for HPC testing, performance, benchmarking (Ryan)
      * - CASA 6 and CASA next generation infrastructure (Ryan)
   * 10:45 - Discussion Session: Open previous CASA follow-on
   * 11:15 - <span class="WYSIWYG_COLOR" style="color: royalblue;">Coffee break</span>
   * 11:30 - Discussion topic: Unique SPW IDs also across OUS (Felix/all)
   * 12:00 - Discussion topic: Use of EU cluster machines for CASA/PL development, best processes (Luke/all)
   * 12:30 - Lunch
   * 13:45 - Pipeline plans for Cycles 7, 8 and beyond (Kana)
   * 14:30 - [[%ATTACHURLPATH%/Remy_PLF2F_summary.pdf][Summary of decisions from Pipeline]] <a href="https://www.eso.org/projects/alma/arc/tw/bin/edit/External/F2F?topicparent=External.ESOCASAPipelineMeeting2019;nowysiwyg=0" title="F2F (this topic does not yet exist; you can create it)">F2F</a> in March 2019 (Remy)
   * 15:30 - <span class="WYSIWYG_COLOR" style="color: royalblue;">Coffee break</span>
   * 15:45 - [[%ATTACHURLPATH%/LTM_EU_PLWG_activities.pdf][Status of EU PLWG]] (Luke)
   * 16:15 - [[%ATTACHURLPATH%/DAREDandADAPT.pdf][DARED and ADAPT]] (Maurizio)
   * 17:00 - CLAI - Cluster/ADAPT Interface (Alex)
   * 17:15 - Information topic: Flux Service (Alisdair)
   * *19:30 Dinner at Paulaner Bräuhaus - See information and map above*
---++++ Friday, 17 May
   * 9:30 - <font size="2">QA2 at the EU ARC</font> (Dirk P.) & Operations Pipeline at the EU ARC (Vinod)
   * 10:30 - Discussion topic: Product ingestion problems how to use the Pipeline for manual data-reduction? (Felix/all)
   * 11:00 - <span class="WYSIWYG_COLOR" style="color: royalblue;">Coffee break</span>
   * 11:15 - Discussion of EU development possibilities for better user products/choices (Luke)
   * 12:00 - Lunch
   * 13:00 - AQUA (Alex and Suzanna)
   * 13:30 - APA - AQUA Pipeline Agent and its Future (Fabrizia)
   * 14:00 - Discussion topic: ESO QA changes for Cycle 7 and beyond with the introduction of ADAPT (Requested by Kana)
   * 14:30 - Information topic: Antenna positions service (Federico)
   * 14:45 - Pipeline Scores (John H. via VC)
   * 15:15 - <span class="WYSIWYG_COLOR" style="color: royalblue;">Coffee break</span>
   * 15:30 - Open discussion (all)
   * End of meeting
---++ 6. Summary of discussions

---++ 7. Action items

---++ 8. Input for Topics to Discuss

   * Pipeline
      * Pipeline: status and plans for Cycles 7, 8 and beyond (Kana)
      * An overview of flagging in pipeline stages (TBD)
      * Overview of the Pipeline and how it works (Stewart W.)
      * Pipeline scores (TBD)
      * Pipeline: imaging (Dirk M.)
      * Pipeline: weblogs, content, how, where, and what connections. Tips and Tricks (Stewart W.)
      * Pipeline failures; MPI timeouts and what they mean (Federico)
      * Pipeline development process and the Integration and Performance team (Ryan, Sandra)
   * CASA
      * CASA: status and plans (Ryan)
      * CASA 6, CASA next generation infrastructure (Ryan)
      * CASA forward plan for HPC testing, performance, benchmarking (Ryan)
      * CASA and Pipeline: status of parallelisation and performance (Federico/Sandra)
      * Ways forward on Virtual-columns+lazy import: how to be able to deliver calibrated MS to all users without having to store them
   * ESO QA2 and ingestion
      * QA2 overview (Dirk P.)
      * QA2 manual processing
         * Python 3 migration plans
         * Parallelisation of manual runs
      * QA.... (Vinod)
      * QA - how the ARC nodes work (TBD)
      * ESO QA changes for Cycle 7 and beyond with the introduction of ADAPT
      * Product ingestion problems: Can the PL tasks be used for manual data-reduction? (Felix/all)
   * Others
      * Summary of decisions from Pipeline F2F in March 2019
      * Status of EU PLWG (Luke)
      * Unique SPW IDs also across OUS: should we make a request to control? (Felix/all)
      * Group-level Processing, how to make best progress (Felix/all)
      * Discussion of EU development possibilities for better user products/choices
      * Use of EU cluster machines for CASA/PL development, best processes
   * Software specific
      * AQUA (Alex and Suzanna)
      * APA (AQUA Pipeline Agent) (Fabrizia)
      * DARED and ADAPT (Maurizio)
      * Flux Service (Alisdair)
      * Antenna positions service (Federico)

-- %USERSIG{SandraCastro - 2019-03-22}%

   * [[%ATTACHURLPATH%/Interferometry-Imaging-Status-EU-ARC-May-2019.pdf][Interferometry-Imaging-Status-EU-ARC-May-2019.pdf]]: Interferometry Imaging Status

   * [[%ATTACHURLPATH%/Pipeline_Overview.pdf][Pipeline_Overview.pdf]]: Pipeline Overview

   * [[%ATTACHURLPATH%/eso_casa_pl_201905_parallelization_performance.pdf][eso_casa_pl_201905_parallelization_performance.pdf]]: CASA Parallelisation and Performance

   * [[%ATTACHURLPATH%/CASA_Status_201905.pdf][CASA_Status_201905.pdf]]: Pipeline Development and the Integration and Performance with CASA

   * [[%ATTACHURLPATH%/CASA_Status_and_FowardPlan201905.pdf][CASA_Status_and_FowardPlan201905.pdf]]: CASA Status and Forward Plan

   * [[%ATTACHURLPATH%/Remy_PLF2F_summary.pdf][Remy_PLF2F_summary.pdf]]: Remy_PLF2F_summary.pdf

   * [[%ATTACHURLPATH%/DAREDandADAPT.pdf][DAREDandADAPT.pdf]]: DARED and ADAPT

   * [[%ATTACHURLPATH%/LTM_EU_PLWG_activities.pdf][LTM_EU_PLWG_activities.pdf]]: LTM_EU_PLWG_activities.pdf

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
pdf	APA_ESOPL16052019_FG.pdf	r1	manage	38400.5 K	2019-05-16 - 13:48	FabriziaGuglielmetti
pdf	CASA_Status_201905.pdf	r1	manage	93.9 K	2019-05-16 - 14:31	SandraCastro	Pipeline Development and the Integration and Performance with CASA
pdf	CASA_Status_and_FowardPlan201905.pdf	r1	manage	305.9 K	2019-05-16 - 14:32	SandraCastro	CASA Status and Forward Plan
pdf	DAREDandADAPT.pdf	r1	manage	1877.4 K	2019-05-16 - 15:33	SandraCastro	DARED and ADAPT
pdf	Interferometry-Imaging-Status-EU-ARC-May-2019.pdf	r1	manage	768.7 K	2019-05-15 - 10:03	SandraCastro	Interferometry Imaging Status
pdf	LTM_EU_PLWG_activities.pdf	r1	manage	13109.2 K	2019-05-16 - 16:36	LukeMaud
pdf	PaulanerBraeuhausMap.pdf	r1	manage	1542.5 K	2019-04-30 - 16:04	SandraCastro	Paulaner Braeuhaus
pdf	Pipeline_Overview.pdf	r1	manage	4901.1 K	2019-05-15 - 10:04	SandraCastro	Pipeline Overview
pdf	Remy_PLF2F_summary.pdf	r1	manage	285.6 K	2019-05-16 - 15:30	LukeMaud
pdf	eso_casa_pl_201905_parallelization_performance.pdf	r1	manage	1210.4 K	2019-05-15 - 11:44	SandraCastro	CASA Parallelisation and Performance

Topic revision: r49 - 2019-05-16 - LukeMaud

External