ARC TWiki> External Web>ESOCASAPipelineMeeting2019 (revision 17)Raw edit | EditAttach
---+ ESO CASA and Pipeline Meeting - 15~17 May 2019


1. Travel to ESO

2. Participants

  • External:
    • Kana Sugimoto (NRAO - NM)
    • Ryan Raba (NRAO - CV),
    • Dirk Muders (MPIFR - Bonn),
    • Stewart Williams (UKRI-STFC - Edinburgh)
  • ESO:
    • Vinod Arumugam
    • Andy Biggs,
    • Sandra Castro,
    • Alex Chalevin
    • Maurizio Chavan,
    • Maria Diaz,
    • Enrique Garcia,
    • Justo Gonzalez,
    • Fabrizia Guglielmetti,
    • Liz Humphreys,
    • Luke Maud,
    • Stewart Mclay,
    • Federico Montesino,
    • Tony Mroczkowski,
    • Dirk Petry,
    • Suzanna Randall,
    • Erich Schmid
    • Felix Stoher,
    • Martin Zwaan
  • Remote participants via VC:
    • ....

3. Video Connection and Meeting Rooms

4. Input for Topics to Discuss

Send us by email any other topic you would like to discuss

  • Pipeline
    • Pipeline: status and plans for Cycles 7, 8 and beyond (Kana)
    • An overview of flagging in pipeline stages (TBD)
    • Overview of the Pipeline and how it works (Stewart W.)
    • Pipeline scores (TBD)
    • Pipeline: imaging and auto-boxing (Dirk M.)
    • Pipeline: weblogs, content, how, where, and what connections. Tips and Tricks (Stewart W.)
    • Pipeline failures; MPI timeouts and what they mean (Federico)
    • Pipeline development process and the Integration and Performance team (Ryan, Sandra)
    • Manifest File (Felix/Stewart M./??)
  • CASA
    • CASA: status and plans (Ryan)
    • CASA 6, CASA next generation infrastructure (Ryan)
    • CASA forward plan for HPC testing, performance, benchmarking (Ryan)
    • CASA and Pipeline: status of parallelisation and performance (Federico/Sandra)
    • Ways forward on Virtual-columns+lazy import: how to be able to deliver calibrated MS to all users without having to store them
  • ESO QA2 and ingestion
    • QA2 overview (Dirk P.)
    • QA2 manual processing
      • Python 3 migration plans
      • Parallelisation of manual runs
    • QA.... (Vinod)
    • QA - how the ARC nodes work (TBD)
    • ESO QA changes for Cycle 7 and beyond with the introduction of ADAPT
    • Product ingestion problems: Can the PL tasks be used for manual data-reduction? (Felix/all)
  • Others
    • Summary of decisions from Pipeline F2F in March 2019
    • Status of EU PLWG (Luke)
    • Unique SPW IDs also across OUS: should we make a request to control? (Felix/all)
    • Group-level Processing, how to make best progress (Felix/all)
    • Discussion of EU development possibilities for better user products/choices
    • Use of EU cluster machines for CASA/PL development, best processes
  • Software specific
    • AQUA (Alex and Suzanna)
    • APA (AQUA Pipeline Agent) (Fabrizia)
    • DARED and ADAPT (Maurizio)
    • Flux Service (Alisdair)
    • Antenna positions service (Federico)

5. Agenda - Draft - Draft

Meeting room: Pavo, A.2.01

Wednesday, 15 May

  • 9:30 - Welcome and introductions (Sandra/Luke)
  • 10:00 - Pipeline status today (Kana)
  • 10:30 - Overview of the Pipeline and how it works (Stewart W.)
  • 11:00 - Pipeline: imaging and auto-boxing (Dirk M.)
  • 11:30 - Coffee break
  • 11:45 - Pipeline: weblogs, content, how, where, and what connections. Tips and Tricks (Stewart W.)
  • TBD - Manifest File (Felix/Stewart M./??)
  • 12:30 - Lunch
  • 14:00 - Pipeline development process and the Integration and Performance team (Ryan/Sandra)
  • 14:30 - Pipeline failures; MPI timeouts and what they mean (Federico)
  • 15:15 - Coffee break
  • 16:00 - CASA and Pipeline: status of parallelisation and performance (Federico/Sandra)
    • should we combine the above two topics?
  • Breze & Beer or Cheese & Wine or Biergarten in Garching (to be decided)

Thursday, 16 May

  • 9:30 - CASA status (Ryan)
  • 10:00 - CASA forward plan for HPC testing, performance, benchmarking (Ryan)
  • 10:30 - CASA 6 and CASA next generation infrastructure (Ryan)
  • 11:30 - Coffee break
  • 11:45 - Ways forward on Virtual-columns+lazy import: how to be able to deliver calibrated MS to all users without having to store them (??)
  • 12:30 - Lunch
  • 14:00 - Summary of decisions from Pipeline F2F in March 2019 (Luke/Kana)
  • 14:30 - Status of EU PLWG (Luke)
  • 15:00 - Pipeline plans for Cycles 7, 8 and beyond (Kana)
  • 15:45 - Coffee break
  • 16:00 - AQUA (Alex and Suzanna)
  • 16:30 - APA (AQUA Pipeline Agent) (Fabrizia)
  • Dinner at TBD

Friday, 17 May

  • 9:30 - ESO QA2 overview (Dirk P.)
  • 10:00 - QA2 Manual Processing (??)
  • 10:30 - Product ingestion problems: Can the PL tasks be used for manual data-reduction? (Felix/all)
  • 11:15 - Coffee break
  • 11:30 - QA - how the EU ARC nodes work (TBD)
  • 12:00 - DARED and ADAPT (Maurizio)
  • ESO QA changes for Cycle 7 and beyond with the introduction of ADAPT
  • Lunch

  • Unique SPW IDs also across OUS: should we make a request to control? (Felix/all)
  • Group-level Processing, how to make best progress (Felix/all)
  • Antenna positions service (Federico)
  • End of meeting

6. Summary of discussions

7. Action items

-- Sandra Castro - 2019-03-22

Comments


Edit | Attach | Watch | Print version | History: r55 | r19 < r18 < r17 < r16 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r17 - 2019-04-24 - SandraCastro
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding ARC TWiki? Send feedback