Getting Started¶
Log-in¶
Login to a standard ELT machine.
IFW Software¶
If not yet done, retrieve and install the complete ICS Framework from the ESO RPMs repository. For more details, please have a look to the installation procedure here
You should create your software based on the provided template by following the procedure in the Getting Started guide here
This guide assumes you have followed all the steps in the above guide.
SysSup Configuration¶
The project template includes a sample of a System Supervisor configuration. After executing the cookiecutter command with the provided template, you can build, install and deploy the provided example.
Please follow the instructions in the general Getting Started guide here
The System Supervisor requires a configuration file including all the definition of all subsystems
managed by the server. A pre-defined configuration has been created as part of the generation
process. This configuration includes five subsystems. They are all considered as internal subsystems.
Generated server configuration: <ins id>/resource/nomad/syssup.yml.tpl. This files contains the Nomad tags so it should not be used directly but through a nomad job command.
!cfg.include config/sup/syssup/server/definitions.yaml:
server: !cfg.type:SysSup
server_id : 'sup'
req_endpoint : "zpb.rr://{{ range service "syssup-req" }}{{ .Address }}:{{ .Port }}{{ end }}/"
pub_endpoint : "zpb.ps://{{ range service "syssup-pub" }}{{ .Address }}:{{ .Port }}{{ end }}/"
db_endpoint : "{{ range service "tinsredis" }}{{ .Address }}:{{ .Port }}{{ end }}"
db_timeout : 2000
scxml : "config/sup/syssup/server/sm.xml"
log_properties : "config/sup/syssup/server/log_properties.cfg"
req_timeout : 120000
dictionaries : []
mon_timeout : 2000
oldb_prefix : 'tins'
ob_modes : [
{
name: Imaging,
subsystems: ['subsim2', 'subsim3']
}
]
subsystems : [
{
name: fcs,
scope: internal,
type: sup::syssup::common::Generic,
rr_endpoint: "zpb.rr://{{ range service "fcs-req" }}{{ .Address }}:{{ .Port }}{{ end }}/StdCmds",
ps_endpoint: "zpb.ps://{{ range service "fcs-pub" }}{{ .Address }}:{{ .Port }}{{ end }}/",
access: true
},
{
name: subsim2,
scope: internal,
type: sup::syssup::common::Generic,
rr_endpoint: "zpb.rr://{{ range service "subsim2-req" }}{{ .Address }}:{{ .Port }}{{ end }}/StdCmds",
ps_endpoint: "zpb.ps://{{ range service "subsim2-pub" }}{{ .Address }}:{{ .Port }}{{ end }}/",
access: true
},
{
name: subsim3,
scope: internal,
type: sup::syssup::common::Generic,
rr_endpoint: "zpb.rr://{{ range service "subsim3-req" }}{{ .Address }}:{{ .Port }}{{ end }}/StdCmds",
ps_endpoint: "zpb.ps://{{ range service "subsim3-pub" }}{{ .Address }}:{{ .Port }}{{ end }}/",
access: true
},
{
name: TestSim1,
scope: internal,
type: sup::syssup::common::Generic,
rr_endpoint: "zpb.rr://{{ range service "tinsccf-req" }}{{ .Address }}:{{ .Port }}{{ end }}/StdCmds",
ps_endpoint: "zpb.ps://{{ range service "tinsccf-pub" }}{{ .Address }}:{{ .Port }}{{ end }}/",
access: true
},
{
name: ocm,
scope: internal,
type: sup::syssup::common::Generic,
rr_endpoint: "zpb.rr://{{ range service "ocm-req" }}{{ .Address }}:{{ .Port }}{{ end }}/std",
ps_endpoint: "zpb.ps://{{ range service "ocm-pub" }}{{ .Address }}:{{ .Port }}{{ end }}/",
access: true
}
]
Note
This configuration shall be adapted to the instrument specific needs.
System Supervisor Logs¶
$ nomad logs -job syssup
The output of the server shall be something like the following:
2021-04-26T09:33:44.931933 INFO Application ocfSupervisor started.
2021-04-26T09:33:44.941777 INFO PS Endpoint: zpb.ps://134.171.3.48:29958/std/status
2021-04-26T09:33:45.047840 INFO Updating DB
2021-04-26T09:33:45.048304 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:31156/StdCmds>
2021-04-26T09:33:45.048465 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:26592/StdCmds>
2021-04-26T09:33:45.048607 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:25925/StdCmds>
2021-04-26T09:33:45.048749 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:22497/StdCmds>
2021-04-26T09:33:45.048892 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:22123/std>
2021-04-26T09:33:45.049533 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:31156/StdCmds>
2021-04-26T09:33:45.050409 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:26592/StdCmds>
2021-04-26T09:33:45.050865 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:25925/StdCmds>
2021-04-26T09:33:45.051282 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:22497/StdCmds>
2021-04-26T09:33:45.051692 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:22123/std>
2021-04-26T09:33:45.086988 INFO min state: -7
2021-04-26T09:33:45.087545 INFO min state: -7
2021-04-26T09:33:45.087728 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:31156/StdCmds>
2021-04-26T09:33:45.087748 INFO [fcs] Generic Connect: zpb.rr://134.171.3.48:31156/StdCmds
2021-04-26T09:33:45.087835 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:26592/StdCmds>
2021-04-26T09:33:45.087848 INFO [subsim2] Generic Connect: zpb.rr://134.171.3.48:26592/StdCmds
2021-04-26T09:33:45.087900 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:25925/StdCmds>
2021-04-26T09:33:45.087913 INFO [subsim3] Generic Connect: zpb.rr://134.171.3.48:25925/StdCmds
2021-04-26T09:33:45.087954 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:22497/StdCmds>
2021-04-26T09:33:45.087967 INFO [TestSim1] Generic Connect: zpb.rr://134.171.3.48:22497/StdCmds
2021-04-26T09:33:45.088001 INFO Subsys RR endpoint - = <zpb.rr://134.171.3.48:22123/std>
2021-04-26T09:33:45.088012 INFO [ocm] Generic Connect: zpb.rr://134.171.3.48:22123/std
2021-04-26T09:33:45.089118 INFO Name: fcs
2021-04-26T09:33:45.089146 INFO Name: subsim2
2021-04-26T09:33:45.089162 INFO Name: subsim3
2021-04-26T09:33:45.089178 INFO Name: TestSim1
2021-04-26T09:33:45.089193 INFO Name: ocm
By default the System Supervisor saves its logs into a file $HOME/supSupervisor.log. A much easier way to see the logs is to monitor this file.
$ tail -f $HOME/supSupervisor
Interacting with the Server¶
The best way to interact with the server is using the SysSup CLI.
$ supcli
zpb.rr://134.171.3.48:27943
supSh>
Note
The shell will connect automatically to the running server.
Getting list of registered subsystems¶
supSh> subsystem_names
reply> = fcs, subsim2, subsim3, TestSim1, ocm
Getting status of subsystems¶
supSh> subsystem_status
reply> = fcs.access = true
fcs.scope = internal
fcs.connection_status = Connected
fcs.state = Operational
fcs.substate = Idle
subsim2.access = true
subsim2.scope = internal
subsim2.connection_status = Connected
subsim2.state = NotOperational
subsim2.substate = NotReady
subsim3.access = true
subsim3.scope = internal
subsim3.connection_status = Connected
subsim3.state = Operational
subsim3.substate = Idle
TestSim1.access = true
TestSim1.scope = internal
TestSim1.connection_status = Connected
TestSim1.state = Operational
TestSim1.substate = Idle
ocm.access = true
ocm.scope = internal
ocm.connection_status = Connected
ocm.state = Operational
ocm.substate = Idle
Restarting an individual subsystem¶
supSh> subsystem_reset subsim3
reply> = OK
supSh> subsystem_init subsim3
reply> = OK init completed.
supSh> subsystem_enable subsim3
reply> = OK enable completed.
Getting global state/substate¶
supSh> status
reply> = Operational;Idle
Changing the access level for a subsystem¶
A subsystem can be ignored by System Supervisor. This can be done by changing its access level to false. When the subsystem is ignored, its state is not considered in the overall state of the system.
supSh> setaccess fcs,false
reply> = OK
supSh> subsystem_status fcs
reply> = fcs.access = false
Getting the actual configuration used by the System Supervisor¶
supSh> get_config
reply> = ...
server: !cfg.type:SysSup
server_id: sup
req_endpoint: zpb.rr://134.171.2.248:28459/
pub_endpoint: zpb.ps://134.171.2.248:20319/
db_endpoint: 134.171.2.248:26498
db_timeout: 2000
scxml: config/sup/syssup/server/sm.xml
log_properties: config/sup/syssup/server/log_properties.cfg
mon_timeout: 2000
req_timeout: 15000
dictionaries: []
oldb_prefix: tins
fits_prefix: ""
ob_modes:
- name: Imaging
subsystems: [subsim2, subsim3]
subsystems:
- name: fcs
scope: internal
type: sup::syssup::common::Generic
rr_endpoint: zpb.rr://134.171.2.248:25124/StdCmds
ps_endpoint: zpb.ps://134.171.2.248:27455/
access: True
- name: subsim2
scope: internal
type: sup::syssup::common::Generic
rr_endpoint: zpb.rr://134.171.2.248:24273/StdCmds
ps_endpoint: zpb.ps://134.171.2.248:25119/
access: true
- name: subsim3
scope: internal
type: sup::syssup::common::Generic
rr_endpoint: zpb.rr://134.171.2.248:27557/StdCmds
ps_endpoint: zpb.ps://134.171.2.248:29982/
access: true
- name: TestSim1
scope: internal
type: sup::syssup::common::Generic
rr_endpoint: zpb.rr://134.171.2.248:21019/StdCmds
ps_endpoint: zpb.ps://134.171.2.248:23543/
access: true
- name: ocm
scope: internal
type: sup::syssup::common::Generic
rr_endpoint: zpb.rr://134.171.2.248:29388/std
ps_endpoint: zpb.ps://134.171.2.248:31381/
access: true
Stopping the Software¶
You can use the startup/shutdown script provide to stop the whole software. For more details please check the general Getting Started guide here
For stopping only the System Supervisor, the nomad CLI can be used.
$ nomad stop syssup