Difference: EUARCPipelineDay (1 vs. 10)

Revision 102014-06-19 - ElizabethHumphreys

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Line: 6 to 6
 

EU ARC Pipeline Day

Changed:
<
<

Location and Contact Information:

>
>

Location and Contact Information

  Date: Wednesday 18 June
Line: 14 to 14
  Location: Vela, Rm 3.04, ESO New Building. Office map: http://www.eso.org/public/about-eso/travel/garching/
Changed:
<
<
Local Participants (limited to 12): Liz, Felix, Dirk, Sebastien, Pavel, Rosita, Anita, Alvaro, Sergio, Markus (perhaps), Paola, Martin
>
>
Local Participants (limited to 12): Liz, Felix, Dirk, Sebastien, Pavel, Rosita, Anita, Alvaro, Sergio, Paola, Martin
 
Changed:
<
<
Remote Participants: Miroslav, Devaky, Wouter, Ivan, George, Adam, ...
>
>
Remote Participants: Czech, Italy, German Nodes
  Remote Screenshare/Telecon Connection Information:
  • Dial-In: 0049 89 307 6834
  • Presentation Sharing Link: http://webconf.vc.dfn.de/vela
  • Enter as a Guest no username or password needed. The web application might request
    to update the flash player please proceed with the update.
Added:
>
>

Link to Talks and Hands-On Sessions

 
Changed:
<
<
Link to Talks and Hands-On Sessions: http://www.eso.org/projects/alma/arc/tw/bin/view/Pipeline/EUARCPipelineDayPassword
>
>
http://www.eso.org/projects/alma/arc/tw/bin/view/Pipeline/EUARCPipelineDayPassword
 

Technical Information for Local and Remote Participants

Revision 92014-06-18 - FelixStoehr

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Line: 105 to 105
 -- ElizabethHumphreys - 13 Jun 2014

META FILEATTACHMENT attachment="pipeline_day_welcome.pdf" attr="" comment="" date="1402910735" name="pipeline_day_welcome.pdf" path="pipeline_day_welcome.pdf" size="124923" stream="pipeline_day_welcome.pdf" tmpFilename="/usr/tmp/CGItemp17291" user="ElizabethHumphreys" version="1"
Added:
>
>
META FILEATTACHMENT attachment="pipelineworkshop_2014_felix.pdf" attr="" comment="" date="1403077424" name="pipelineworkshop_2014_felix.pdf" path="pipelineworkshop_2014_felix.pdf" size="405938" stream="pipelineworkshop_2014_felix.pdf" tmpFilename="/usr/tmp/CGItemp55477" user="FelixStoehr" version="1"

Revision 82014-06-18 - ElizabethHumphreys

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Line: 23 to 23
 
  • Presentation Sharing Link: http://webconf.vc.dfn.de/vela
  • Enter as a Guest no username or password needed. The web application might request
    to update the flash player please proceed with the update.
Changed:
<
<
Link to Talks and Hands-On Sessions:
>
>
Link to Talks and Hands-On Sessions: http://www.eso.org/projects/alma/arc/tw/bin/view/Pipeline/EUARCPipelineDayPassword
 

Technical Information for Local and Remote Participants

Revision 72014-06-17 - ElizabethHumphreys

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Line: 99 to 99
 
  • Using Pipeline "applycal" scripts/PPR to restore rawdata to calibrated MS
  • Adding manual flagging to Pipeline runs
  • Replacing the flux of the flux calibrator
Deleted:
<
<
  • Adding lines to be flagged in the flux calibrator
 
  • Fixing datasets during Pipeline runs
Deleted:
<
<
  • Inserting own calibration tables
 
  • Running Analyzemscal

-- ElizabethHumphreys - 13 Jun 2014

Revision 62014-06-17 - ElizabethHumphreys

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Line: 6 to 6
 

EU ARC Pipeline Day

Added:
>
>

Location and Contact Information:

 
Changed:
<
<
Date: Wednesday 18 June
>
>
Date: Wednesday 18 June
 
Changed:
<
<
Time: 0930-1730 CEST
>
>
Time: 0930-1730 CEST
 
Changed:
<
<
Location: Vela, Rm 3.04, ESO New Building. Office map: http://www.eso.org/public/about-eso/travel/garching/
>
>
Location: Vela, Rm 3.04, ESO New Building. Office map: http://www.eso.org/public/about-eso/travel/garching/
 
Changed:
<
<
Local Participants (limited to 12): Liz, Felix, Dirk, Sebastien, Pavel, Rosita, Anita, Alvaro, Sergio, Markus (perhaps), Paola, Martin
>
>
Local Participants (limited to 12): Liz, Felix, Dirk, Sebastien, Pavel, Rosita, Anita, Alvaro, Sergio, Markus (perhaps), Paola, Martin
 
Changed:
<
<
Remote Participants: Miroslav, Devaky, Wouter, Ivan, George, Adam, ...
>
>
Remote Participants: Miroslav, Devaky, Wouter, Ivan, George, Adam, ...

Remote Screenshare/Telecon Connection Information:

  • Dial-In: 0049 89 307 6834
  • Presentation Sharing Link: http://webconf.vc.dfn.de/vela
  • Enter as a Guest no username or password needed. The web application might request
    to update the flash player please proceed with the update.

Link to Talks and Hands-On Sessions:

 

Technical Information for Local and Remote Participants

Line: 31 to 39
 

Steps to access the processing cluster

Changed:
<
<
1) Remote user only: ssh into ESO **USE THE TEMPORARY ESO ACCOUNT HERE - this account information has been sent to Wouter (Nordic), George (UK), Remo (Allegro), Jan (Bologna), Miroslav (Czech), Stefanie (Bonn), Frederic (IRAM)
>
>
1) Remote/wireless ESO user only: ssh into ESO
 
ssh <username>@ssh.eso.org

2) Remote and local users: ssh into the cluster gateway arcp1

Line: 65 to 73
 Go to your work area (the home directory should not contain large amounts of data, type
work 
Deleted:
<
<

Screen sharing

Link: http://webconf.vc.dfn.de/vela
Enter as a Guest no username or password needed. The web application might request to update the flash player please proceed with the update.

Telecon Information

ADD NUMBER

 

Agenda

Changed:
<
<
0930-0945 Welcome and Scope (Liz)
>
>
0930-0945 Welcome and Scope (Liz)
  0945-1030 Pipeline Overview (Liz)
Line: 94 to 95
 
  • Using Pipeline to calibrate data from PPR and from python script
  • Editing the PPR
Changed:
<
<
  • Examining the WebLog and Pipeline Products
>
>
  • Examining the WebLog and Pipeline Products
 
  • Using Pipeline "applycal" scripts/PPR to restore rawdata to calibrated MS
  • Adding manual flagging to Pipeline runs
  • Replacing the flux of the flux calibrator

Revision 52014-06-16 - FelixStoehr

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Line: 40 to 40
 3) Start your vncserver on the cluster using the geometry of your screen, e.g.:
getvnc.sh -geometry 2000x1000
Changed:
<
<
4) You will be asked to provide a new password, enter the password of the almaXX ssh account.
>
>
4) You will be asked to provide a new password (you'll have to type it two times), enter the password of the almaXX ssh account.
 
Changed:
<
<
5) On your laptop/home machine, open two terminal windows and copy the the first connection string written out by the script into the first one. Enter the appropriate passwords
>
>
5) On your laptop/home machine, open two terminal windows and copy the the first connection string written out by the script into the first one.

NOTE that you will have to change the first occurrence of "almaXX" with your ssh account (e.g. "jbrand"). The script silently assumes that those two account names are the same which is true for everyone except for this tutorial.

Enter the appropriate passwords

  6) Linux users: use the second connection string to connect to your VNC server

Revision 42014-06-16 - ElizabethHumphreys

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Line: 66 to 66
 Link: http://webconf.vc.dfn.de/vela
Enter as a Guest no username or password needed. The web application might request to update the flash player please proceed with the update.
Telecon Information
Changed:
<
<
Access to the cluster

We have set up temporary ESO accounts for 1 person from each ARC Node, so that remote participants can also join the afternoon session if wanted.

The people with accounts are: George (UK), Frederic (IRAM), Wouter (Nordic), Stefanie (Bonn), Miroslav (Czech), Remo (Allegro), Jan (Italy)

>
>
ADD NUMBER
 

Agenda

Revision 32014-06-16 - ElizabethHumphreys

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Line: 9 to 9
  Date: Wednesday 18 June
Changed:
<
<
Time: 0930-1730
>
>
Time: 0930-1730 CEST
 
Changed:
<
<
Location: Vela, Rm 3.04, ESO New Building
>
>
Location: Vela, Rm 3.04, ESO New Building. Office map: http://www.eso.org/public/about-eso/travel/garching/
 
Changed:
<
<
Local Participants: Liz, Felix, Dirk, Sebastien, Pavel, Rosita, Anita, Alvaro, Sergio, ....
>
>
Local Participants (limited to 12): Liz, Felix, Dirk, Sebastien, Pavel, Rosita, Anita, Alvaro, Sergio, Markus (perhaps), Paola, Martin
 
Changed:
<
<
Remote Participants: Miroslav, Wouter, Ivan, George, Adam, ...
>
>
Remote Participants: Miroslav, Devaky, Wouter, Ivan, George, Adam, ...
 

Technical Information for Local and Remote Participants

Line: 31 to 31
 

Steps to access the processing cluster

Changed:
<
<
1) Remote user: ssh into ESO
>
>
1) Remote user only: ssh into ESO **USE THE TEMPORARY ESO ACCOUNT HERE - this account information has been sent to Wouter (Nordic), George (UK), Remo (Allegro), Jan (Bologna), Miroslav (Czech), Stefanie (Bonn), Frederic (IRAM)
 
ssh <username>@ssh.eso.org
Changed:
<
<
2) Remote and local users: ssh into the cluster gateway arcp1
>
>
2) Remote and local users: ssh into the cluster gateway arcp1
 
ssh <username almaXX>@arcp1.hq.eso.org

3) Start your vncserver on the cluster using the geometry of your screen, e.g.:

Line: 74 to 74
 

Agenda

Changed:
<
<
0930-0945 Welcome and Scope (Liz)
>
>
0930-0945 Welcome and Scope (Liz)
  0945-1030 Pipeline Overview (Liz)
Line: 104 to 104
 
  • Running Analyzemscal

-- ElizabethHumphreys - 13 Jun 2014

Added:
>
>
META FILEATTACHMENT attachment="pipeline_day_welcome.pdf" attr="" comment="" date="1402910735" name="pipeline_day_welcome.pdf" path="pipeline_day_welcome.pdf" size="124923" stream="pipeline_day_welcome.pdf" tmpFilename="/usr/tmp/CGItemp17291" user="ElizabethHumphreys" version="1"

Revision 22014-06-13 - FelixStoehr

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Line: 18 to 18
 Remote Participants: Miroslav, Wouter, Ivan, George, Adam, ...

Technical Information for Local and Remote Participants

Deleted:
<
<

Local

 
Changed:
<
<

Remote

Screen sharing
>
>
Requirements: install a VNC client on your laptop/home system if there is not already one installed (on Linux, type
vncviewer --help

to check).

Red Hat:

sudo yum install tigervnc  
Ubuntu:
sudo apt-get install xtightvncviewer

MacOS: e.g. Chicken of the VNC.

Steps to access the processing cluster

1) Remote user: ssh into ESO

ssh <username>@ssh.eso.org

2) Remote and local users: ssh into the cluster gateway arcp1

ssh <username almaXX>@arcp1.hq.eso.org

3) Start your vncserver on the cluster using the geometry of your screen, e.g.:

getvnc.sh -geometry 2000x1000

4) You will be asked to provide a new password, enter the password of the almaXX ssh account.

5) On your laptop/home machine, open two terminal windows and copy the the first connection string written out by the script into the first one. Enter the appropriate passwords

6) Linux users: use the second connection string to connect to your VNC server

7) MacOS users: start Chicken of the VNC and connect to "localhost".

You should see a virtual terminal on the processing cluster.

8) At the end of the day, remove the vnc server from the cluster with

getvnc.sh -k

Working on the cluster

Get an interactive shell one the freest cluster node (the gateway node should not be used for processing), type

qi

Go to your work area (the home directory should not contain large amounts of data, type

work 

Screen sharing

  Link: http://webconf.vc.dfn.de/vela
Enter as a Guest no username or password needed. The web application might request to update the flash player please proceed with the update.
Telecon Information

Revision 12014-06-13 - ElizabethHumphreys

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="WebHome"

EU ARC Pipeline Day

Date: Wednesday 18 June

Time: 0930-1730

Location: Vela, Rm 3.04, ESO New Building

Local Participants: Liz, Felix, Dirk, Sebastien, Pavel, Rosita, Anita, Alvaro, Sergio, ....

Remote Participants: Miroslav, Wouter, Ivan, George, Adam, ...

Technical Information for Local and Remote Participants

Local

Remote

Screen sharing

Link: http://webconf.vc.dfn.de/vela
Enter as a Guest no username or password needed. The web application might request to update the flash player please proceed with the update.

Telecon Information

Access to the cluster

We have set up temporary ESO accounts for 1 person from each ARC Node, so that remote participants can also join the afternoon session if wanted.

The people with accounts are: George (UK), Frederic (IRAM), Wouter (Nordic), Stefanie (Bonn), Miroslav (Czech), Remo (Allegro), Jan (Italy)

Agenda

0930-0945 Welcome and Scope (Liz)

0945-1030 Pipeline Overview (Liz)

1030-1115 Pipeline Heuristics (Liz)

1115-1130 Coffee

1130-1145 Comparison of manual and Pipeline reductions (Liz)

1145-1200 Pipeline Products and Archive (Felix)

1200-1230 Pipeline in QA2 (Dirk)

Lunch

1330-1730 Hands on work and discussion

  • Using Pipeline to calibrate data from PPR and from python script
  • Editing the PPR
  • Examining the WebLog and Pipeline Products
  • Using Pipeline "applycal" scripts/PPR to restore rawdata to calibrated MS
  • Adding manual flagging to Pipeline runs
  • Replacing the flux of the flux calibrator
  • Adding lines to be flagged in the flux calibrator
  • Fixing datasets during Pipeline runs
  • Inserting own calibration tables
  • Running Analyzemscal

-- ElizabethHumphreys - 13 Jun 2014

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding ARC TWiki? Send feedback