Table of Contents

• Day 1
  
  • Session 1: Overview
  • Session 2: CPU Platforms
    
  • Session 3: ELT Projects
  • Session 4: VLT Projects
  • Session 5: Non-ESO Projects
• Day 2
  
  • Session 6: GPU Platform
  • Session 7: Heterogenous Platforms
  • Session 8: ELT Projects
  • Session 9: VLT Projects
  • Session 10: Other ESO Projects
• Day 3
  
  • Session 11: Machine Learning
  • Session 12: Algorithms
  • Session 13: Other Developments

Day 1

Introduction (B. Jeram, ESO)

Welcome (S. Ramsey, ESO)

INDUSTRY KEYNOTE TALK: Advancing Efficiency, an AMD Overview (M. Foley, AMD)

AMD has established a significant performance leadership position in server CPUs. In a time where sustainability and efficiency are increasingly important, an optimal package of performance and efficiency is a solid cornerstone for success.

In this talk we will cover the technology considerations which influenced the development of these products, as well as provide a discussion of technology trends which will shape the future of the server infrastructure market.

    Session 1: Overview

Next Generation Real Time Computers at ESO (M. Suárez Valles, ESO)

Driven by the ELT requirements, AO RTC at ESO has shifted from dedicated hardware platforms towards networked, time-synchronized, high-performance computing systems. Enabling CPU platforms and software paradigms were initially prototyped within a external contract with FORCE Technology in 2018-2020 and demonstrated to scale up to the ELT MCAO size. Since then, the core techniques have been further evolved at ESO and applied to new projects such as the VLT SPARTA Upgrade and the ELT Wave Front RTC. Common approaches and tools are being developed across ESO projects to maximize performance determinism on CPU platforms.

These include streamlining OS and network tuning, resource affinity, recurrent lockless scenarios, performance verification on-the-wire, etc. This talk presents the main drivers for the new AO RTC paradigm adopted by ESO, its evolution since the initial prototyping and the core technologies currently in place. The performance envelope attainable so far is summarized, along with the current approach to performance verification and observability.

Possible future evolution and obsolescence concerns are discussed.

RTC Toolkit - Overview and Status (B. Jeram, ESO)

The RTC Toolkit (RTC Tk) is a collection of software tools, libraries, and example implementations designed for developing the Soft Real-Time Cluster (SRTC) of an ELT-scale adaptive optics real-time computer (AO RTC). This presentation will provide a brief overview of the RTC Toolkit, highlighting its features and discussing its current development status.

    Session 2: CPU Platforms

HEART System Architecture: A Brief Overview (Malcolm Smith, Herzberg Astronomy and Astrophysics) 

The Herzberg Extensible Adaptive optics Real-Time Toolkit (HEART) is currently under development at Herzberg Astronomy and Astrophysics in Victoria.

HEART is a CPU based design which is scalable from modest sized systems to very large future AO systems such as the TMT NFIRAOS.  The system architecture decomposes an RTC system into multiple HEART pipes consisting of one or more HEART blocks.  Each HEART pipe is a collection of blocks which perform processing of one or more data streams. Blocks are connected within the pipe in such a way that downstream blocks process the outputs of the upstream block. These blocks may be triggered sequentially within the block or may operate on a data stream in parallel in a pipelined fashion.

Different pipe and block configurations allow HEART to be used to implement a variety of different types of AO systems.

This presentation will provide a high level overview of the architectural design of HEART along with information about how HEART is being used to develop several different RTC system.

HEART On-Sky Results and Soft-Realtime Functionality (J. Dunn, Herzberg Astronomy and Astrophysics)

The Herzberg Extensible Adaptive Real-time Toolkit (HEART) is a collection of libraries and other software components developed at NRC-HAA that can be used to control different types of Adaptive Optics (AO) systems.

The HEART RTC design has two main components: the Hard Real-Time RTC (HRT), which include all the time-critical high-speed processing tasks; and the Soft Real-Time RTC (SRT), which include the non-time critical tasks such as optimizations and reconstruction generation.

This talk will cover on-sky results from a closed loop and open loop run. It will also touch on the functionality provided by the soft-realtime.

HEART build and test infrastructure (E. Chapin, Herzberg Astronomy and Astrophysics)

The Herzberg Extensible Adaptive optics Real-Time Toolkit (HEART) is a highly versatile framework now incorporated into the designs of RTCs for a number of observatories (including test benches at the Dominion Astrophysical Observatory, and facility instruments for Gemini, TMT, and ELT).

In order to manage development of its complex C and Python code base, and to support ongoing projects that target multiple architectures and varying requirements, a robust build and test system was required.

In this presentation I will summarize our team's strategy for revision control, continuous integration, and an overview of our low-level unit and higher-level black box component and integration tests. Our system has allowed us to refactor and extend our code extensively as we have taken on new projects.

ALPAO RTC Performances and new features (B. Martin, ALPAO)

Real-Time Computing (RTC) is an important challenge in Adaptive Optics (AO) in order to compensate for high-velocity perturbations. The ultimate performance is often a trade-off between expensive and complex clusters and versatility. ALPAO has developed the ALPAO RTC solution, which runs on a standard server, making it highly adaptable to various AO system configurations.

It can run on a common server machine or a common cluster. Originally, ALPAO RTC was designed to control AO loops involving Shack-Hartmann Wave Front Sensors. ALPAO RTC is also part of SISO (Single Input Single Output) and MIMO (Multiple Input Multiple Output) systems that achieve, for instance, a frame rate of 3kHz and an RTC latency delay (the duration between total retrieval of the image and command sent) of less than 70µs on a 256x256 pixels, 32x32 micro-lens SHWFS, and an 820-actuator Deformable Mirror. ALPAO RTC is also running in a Laser Guide Star system, controlling downlink and uplink.

Finally, ALPAO RTC is now part of a system involving a Pyramid Wave Front Sensor currently operating on a telescope.

HRTC[p]: ELT-sized hard realtime core on common-off-the-shelf hardware (N-H. Pedersen, FORCE Technology and P-H Kamp, The FreeBSD Project)

In response to the challenging requirements of the ELT First Light Instruments RTCs, ESO triggered in 2018 an external prototyping contract with FORCE Technology as part of a wider package of RTC de-risking activities.

At the core of this development was the identification of CPU-based hardware configurations, along with software and communication paradigms enabling RTC deterministic performance while preserving reasonable long-term maintainability and obsolescence resilience.

The result hereof, the HRTCp (Hard Real Time Core prototype), is a full scale prototype consisting of standard off-the-shelf PC servers and networking.

Using Linux as a hardware adaptation layer, HRTCp implements ELT MCAO-size pseudo open-loop control (six LGS, 500 Hz frame rate), from decoding sensor image data to controlling the ELT M4 and M5 mirrors.

The talk presents the system architecture and points out some of the more interesting challenges and features of the resulting software system.

Results from operating the HRTCp in an environment with externally simulated sensor- and mirror systems are presented.

INDUSTRY KEYNOTE TALK:  Fast DDS - The middleware powering the ESO ELT (J. Martin Losa, eProsima)

ESO has selected fast DDS as the middleware for the Extremely Large Telescope (ELT) Control System. DDS is the middleware behind the Core Integration Infrastructure Middleware Abstraction Layer (CII MAL) and the RTC Toolkit for propagating Telemetry data. DDS stands for Data Distribution Service for real-time systems, and it is a standard of the OMG for real-time publish-subscribe middleware.

This presentation will introduce the DDS standard and the benefits of eProsima’s Fast DDS implementation.

    Session 3: ELT Projects

MORFEO RTC status report (I. Foppiani,  INAF-OAS)

This talk presents a report on the design of the RTC for MORFEO, the MCAO module for the ESO Extremely Large Telescope. MORFEO provides extensive sky coverage, wide field of view and diffraction-limited correction in the near infrared. Its wavefront sensing is based on six Laser Guide Stars (LGS) and up to three Natural Guide Stars (NGS) for tomographic atmospheric turbulence reconstruction, while three deformable mirrors (ELT M4 and two post-focal DMs) provide wavefront correction on a field up to 2’.

The core control strategy of MORFEO is the Pseudo Open Loop Control (POLC) implemented by means of three control loops: the High Order (HO) loop running at 500 Hz, the Low Order (LO) loop running at 0.1-1 Hz, and the Reference (REF) loop operating at 0.1-100 Hz. The HO and LO loops handle split tomography and real-time DMs control, while the REF loop corrects slow evolving aberrations due both to the system and to the atmosphere. The three control loops are implemented in the hard real time core of the RTC, the HRTC, based on a hardware design focused on low latency and low jitter.

The soft RTC (SRTC), based on the RTC-Toolkit provided by ESO, performs supervisory, optimization, measurements and calibration tasks. INAF supervises the overall development of the RTC and in particular of the SRTC. NRC-HAA, having recently joined the MORFEO Consortium, is in charge of the final design and deployment of the HRTC.

The SCAO module for ANDES, the high resolution spectrograph for the ELT (P. Di Marcantonio, INAF)

ANDES (ArmazoNes high Dispersion Echelle Spectrograph), formerly known as ELT-HIRES, is the high-resolution optical-infrared spectrograph for the ESO/ELT (European Southern Observatory/Extremely Large Telescope) thought to study astronomical objects that require highly sensitive observations. It will be used to search for signs of life in Earth-like exoplanets, find the first stars born in the Universe, test for variations of the fundamental constants of physics, and measure the acceleration of the Universe's expansion.

For the biosignature detection scientific case, ANDES will be able to directly probe exoplanets, by spatially resolving them from their host star, focusing on their reflected star light and taking advantage of the angular resolution of the ELT with AO-assisted observations. To this purpose it will be equipped with a dedicated single conjugate adaptive optics (SCAO) module.

In this talk I’ll briefly present first the ANDES instrument and its project, focusing afterwards on the ANDES SCAO subsystem presenting its capabilities and challenges.

    Session 4: VLT Projects

SPARTA Upgrade (P. Shchekaturov, ESO)

SPARTA Upgrade is the obsolescence project to upgrade an old legacy SPARTA hard-real time box by the modern technology HW and innovated SW using new standards and advanced technics.

The project was triggered by the red flag from ESO RTC development team raising the awareness that some of the Paranal SPARTA systems might not fulfill the lifetime of their instruments lifetime (AOF GALACSI, ERIS and etc.) due to the increase failure rate of their HW. The development was inspired by the ELT technology development program, where it was demonstrated that the new modern CPU based servers can provide sufficient performance results, improving the maintenance by using off-the-shelf HW and mitigating the obsolescence issues.

The project development includes the upgrade of the I/O HW to the new modern PCIe cards maintaining sFPDP interface to the existing WFS cameras and DSM, keeping the interface to the soft real-time SPARTA coprocessing cluster unmodified, but replacing the underlying SW of the hard real-time control processing core with two socket AMD based CPU server. Together with the HW upgrade the modern technologies and SW technics were introduced as well like PTP, 10Gb Ethernet, lock-free programming, new C++ standards features and etc.

This new development was also propagated to the new instruments requiring the modern RTC in the timeline of the new VLT instruments like GRAVITY+, where the RTC is based on the SPARTA Upgrade.

GRAVITY+ RTC design (R. Dembet, Observatoire de Paris - LESIA)

GRAVITY+ is an upgrade project of the existing GRAVITY instrument at Paranal with new Adaptive Optics systems and an upgrade of the VLT infrastructure to accommodate one laser on each UT to boost the performance. The project incorporates the upgrade of existing CIAO and MACAO AO systems with bigger DM, faster WFS and more powerful RTC. One LGS system will be installed on each UT as an addition to these two SCAO systems upgrade to increase the sky coverage.

The new RTC is based on SPARTA Upgrade, where the co-processing cluster SW is based on old SPARTA legacy SW, but the HRTC part is based on new upgraded version of SPARTA RTC box HW and updated version of Platform SW. SPARTA upgrade for GRAVITY+ accommodate different type of Ethernet based I/O interfaces like GigE for WFS cameras, custom UDP based protocol for ALPAO DM, TCP IP based protocol for M2 fast tip-tilt offload interface and MUDPI protocol for laser jitter actuators. sFPDP interface is also maintained for CIAO SAPHIRA WFS by using new PCIe cards from SPARTA upgrade. The core hard real-time control SW is partially based on SPARTA Upgrade components, but some control blocks are specifically designed and implemented for GRAVITY+ RTC like collaborative control for fast TT offload, anti-windup filtering, chopping etc. The soft real-time co-processing cluster SW is partially based on the experience from SPARTA for ERIS, but most of the core algorithm are redesigned and implemented particularly for GRAVITY+.

    Session 5: Non-ESO Projects

A Clustered Architecture of Real-Time Controller for 2.5-Meter Solar Telescope (WeHoST) GLAO System (N. Yan, Institute of Optics and Electronics)

A solar Ground Layer Adaptive Optics (GLAO) system was considered for the 2.5-meter wide-filed and high-resolution solar telescope (WeHoST) under construction in China. Five separated correlation Shark-Hartmann wavefront sensors (SH-WFS), including one small FoV, high-order SH-WFS and four large FoV, low-order SH-WFS, and a 931-element deformable mirror are employed to enhance the image quality within 5 arcminutes field of view (FoV). Compared to the center of gravity algorithm, the correlation algorithm demands higher computational resources. To solve this problem, a clustered architecture platform based on multi-core CPUs was employed for the real-time controller (RTC) of the 2.5-meter solar GLAO system. The clustered architecture consists of five slope calculation nodes, each corresponding to one of the five SH-WFS, as well as a control node. Subimages from five SH-WFSs are acquired synchronously and the slopes are calculated in the slope calculation nodes.

The control node collects slopes from the slope calculation nodes and performs the wavefront reconstruction. The clustered architecture platform has the benefit of the computing power and scalability. Moreover, choosing multi-core CPUs reduces hardware design complexity and offers enhanced flexibility. The clustered architecture is currently undergoing testing. This report primarily focuses on the performance analysis of individual node computation latency and inter-node communication latency.

The Architecture of the MCAO Real-time Control Computer Cluster for the DKI Solar Telescope (D. Schmidt, National Solar Observatory)

The real-time control system for the multi-conjugate adaptive optics system for the 4-meter Daniel K. Inouye Solar Telescope will process 11817 subapertures in total, each 20x20 px, received from nine wavefront sensors and control three deformable mirrors with a total of 3452 actuators. The system will run the control loop at a rate of 2000 Hertz. The control system is based on the KAOS Evo 2 software and will run on a cluster of ten x86 Linux servers interconnected with a 200-gigabits-per-second Infiniband network.

We will use the adaptive optics simulation software Blur to test the cluster implementation before deployment at the telescope. Blur will present the cluster with faked observations, simulating both the wavefront sensor images and the deformable mirror effects in the imaging of a solar imaging through turbulence.

At this workshop, we plan to present the software and hardware architecture of the real-time control system, including synchronization techniques we use for lowest latency, motivate our choices and report on the timing performance and the things we have learned from using 200-gigabits-per-second Infiniband.

A CPU based Real-Time control system for Adaptive Optics in Free Space Optical Communications (D. Guerra Ramos, DLR)

Adaptive Optics is a rapidly growing to be an essential technology enabler in the field of Free Space Optical Communications. The DLR Institute for Communications and Navigation has been operating a number of experiment AO systems for a variety of applications in FSOC for over 6 years.

The challenges in FSOC AO are similar but subtly different to astronomical AO. In FSOC high frame rates of over 2kHz and low RTC latencies are required, but the problem size is typically much reduced in comparison to the state-of-the-art systems designed for the upcoming Extremely Large Telescopes. At DLR-KN, a software has been developed to control FSOC AO systems with high frame rates and has been successfully demonstrated in a number of AO systems. The software currently relies only on high performance CPU based computers, though acceleration is also under consideration for future AO systems.

Here, we describe the design drivers and resulting architecture of the Real-Time Control System that powers the DLR AO demonstrators and report on benchmarking results demonstrating its computational performance.

Day 2

INDUSTRY KEYNOTE TALK: Accelerated Computing Infrastructure with NVIDIA (P. Mohan, NVIDIA)

Accelerated computing can enable unprecedented levels of performance and scalability for scientific applications.

This talk covers the advancements in accelerated computing infrastructure with NVIDIA DGX, DPUs and data centers, and how to leverage them to enable your critical HPC workloads.

INDUSTRY KEYNOTE TALK:  FPGAs for flexible interfacing and distributed computational tasks (R. Biasi, Microgate)

Actual implementations of RTCs require connecting to several sensors and actuators with different interfaces, for which low latency data flow is demanded by the real-time pipeline. We developed dedicated FPGA-based solutions allowing abstraction of the HW interface layer and assuring deterministic data transfer with GPUs and CPUs. FPGAs can also offload part of the RTC tasks by implementing efficient distributed parallel computational schemes, as it happens for specific computations required by some current generation large deformable mirrors.

    Session 6: GPU Platforms

COSMIC, an adaptable, powerful, proven RTC solution (Damien Gratadour, Observatoire de Paris)

COSMIC is an ESO compliant, GPU-based RTC platform, to be used as the Real-Time platform in MAVIS and MICADO. Developments have been done in ObsPM LESIA and ANU AITC. In collaboration with Microgate, COSMIC is at the core of several existing RTC, Keck (2021), the ESO-GHOST lab platform (2023). Other incarnations are under development. Leveraging GPUs, the COSMIC s/w stack has evolved in the past years, with implementation flexibility, performance, and reliability in mind (see talk by J.Bernard).

COSMIC: A Graph-Based, Extensible Framework for the Future of Adaptive Optics RTC development (J. Bernard, ANU)

AO demands rapid, real-time computation to counteract atmospheric perturbations. COSMIC emerges as a modern solution, employing a graph-based architecture where operations are intuitively represented as nodes. It aims at simplifying design, implementation, testing and integration by relying on robust concepts and useful tools. Notably, COSMIC harnesses GPUs for accelerated computation and is adept at scaling across multiple processes without overhead using shared memory. Recent updates have further enhanced its versatility, cementing its potential as a future-proof, extensible framework for AO advancements.

COSMIC SRTC: review and future plans (N. Doucet, ANU)

The RTC platform COSMIC envisioned for MAVIS and MICADO provides a SRTC to assist the HRTC and handle computationally demanding tasks that are time constrained. This SRTC is designed to cope with the requirements of the next generation of instruments relying on the computation power of GPUs and state of the art High Performance Computing algorithms and libraries.

High-performance low latency data acquisition for AO RTC (J. Plante, Observatoire de Paris)

Adaptive Optics rely on a closed loop system featuring challenging real-time constraints. In this context, a response time of under few 100 µs is typically targeted. To achieve this extremely low latency, our team at LESIA developed a GPU-based RTC solution relying on the COSMIC framework. In order to receive frames sent by the WFS camera with very low latency, a custom FPGA board was initially developed. In an effort to adopt standard solutions, the same kind of performance could be achieved using advanced features on high end COTS network controllers (NIC).

In this talk, we will present a new solution based on a COTS NIC and the DPDK gpudev library. This technology enables DMA from NIC to GPU, and makes it possible to reach latencies as low as 4.2 +/- 0.5 µs, a very small portion of the typical time budget for a 10 Gb/s link. We start this presentation by an overview of the MICADO RTC, as well as an introduction of other data acquisition strategies. We continue by diving into the details of our solution and how it was adopted as the baseline solution for this instrument. Finally, we open on the implications of this work, and the impact it could have on other RTC such as on VLT-MAVIS leveraging multiple WFS concurrently, as well as in other domains.

Can we overcome the performance/portability tradeoff on GPU pipelines (C. Cetre, Thales Research & Technology)

The tremendous improvement of Graphic Processing Units (GPUs) hardware and software makes them a good fit with real-time Adaptive Optics computation pipelines, although full of challenges. The help of modern GPU architecture and some non-conventional GPU programming techniques makes the sub millisecond requirement no longer impossible.

Unfortunately, GPU programming requires a deep understanding of its mechanisms in order to achieve this kind of performance. It also raises concerns about global maintainability of such systems.

This talk will dive into the GPU mechanisms that facilitate the integration of discrete accelerators into time sensitive system that are in use within the real-time AO control platform COSMIC By taking advantage of its asynchronous nature, we display how it is effectively possible to divert most latencies and overheads from the critical path. We will then discuss possible ways to improve productivity & maintainability of such systems The tremendous improvement of Graphic Processing Units (GPUs) hardware and software makes them a good fit with real-time Adaptive Optics computation pipelines, although full of challenges. The help of modern GPU architecture and some non-conventional GPU programming techniques makes the sub millisecond requirement no longer impossible.

Unfortunately, GPU programming requires a deep understanding of its mechanisms in order to achieve this kind of performance. That is why using a high-level programming model such as OpenMP while keeping the performance obtained on a hardware specific one like CUDA would be a good balance, although OpenMP lacks, so far, the right tools to reach real-time performance. The Barcelona SuperComputing center proposes a novel approach with a compiler transformation technique that turns OpenMP directives into CUDA code, allowing a significant performance increase. This talk will cover the basics of enabling GPU computations for AO real-time control with the COSMIC platform. It will then present how compiler transformation techniques can turn an OpenMP pipeline into a high performance GPU graph without trading-off on performance, which with our contribution to the CLANG compiler gives all the necessary tools to implement AO pipelines in such environment and get a better understanding of the GPU behavior.

    Session 7: Heterogenous Platforms

Durham Adaptive Optics (DAO) real-time controller (S. Cetre, Durham University)

Durham Adaptive Optics (DAO) is a powerful and flexible software solution for adaptive optics systems.  DAO enables real-time correction of wavefront distortions caused by atmospheric turbulence and optical aberrations, improving the image quality of ground-based telescopes. DAO takes a hardware-agnostic approach to processing pipelines, supporting distributed heterogeneous compute environments. Its high flexibility allows seamless integration with various hardware systems and configurations, accommodating different wavefront sensors (such as Shack-Hartmann and pyramid sensors), actuators (including deformable mirrors, tip-tilt mirrors, and spatial light modulators), and other components. We will present the architecture of DAO and how it can be used as the building blocks for AO systems of any size. From, fully software simulation, lab bench systems all the way to ELT scale systems such as HARMONI and MOSAIC.

The presentation will cover the software's flexible architecture, which enables it to be integrated with a variety of hardware systems and configurations. We will showcase DAO’s user base and how DAO has been used to solve their adaptive optics real-time control needs. These examples will demonstrate DAO’s efficient data handling, parallel processing techniques, low latency, and minimal jitter, whilst emphasising its capacity to scale to AO systems of all size, from laboratory-based research projects to ELT-scale facility class systems.

    Session 8: ELT Projects

MICADO SCAO RTC: building the ELT first light RTC (F. Ferreira, CNRS)

The SCAO RTC of the first light ELT instrument MICADO has successfully passed its final design review. The RTC has actively begun the manufacturing, assembly, integration and verification phase. MICADO SCAO RTC hard real-time capabilities are provided by COSMIC, while the soft real-time features rely on the ESO RTC Toolkit. This talk will present the MICADO SCAO RTC architecture, the design choices and the current status of the project.

It will provide an overview of the H-RTC pipeline design, including performance benchmark and validation through simulation mode and on-bench tests. It will also picture the current status of the RTC Toolkit integration activities for the S-RTC on multi-nodes cluster including the first implementation of telemetry consumption, data tasks, data storage, data visualization, and H-RTC optimization.

The METIS AO RTC at the beginning of MAIT (M. Kulas, MPI)

AMETIS is one of the 1st generation instruments of the ELT. Its AO system is equipped with an RTC that uses COTS components and conforms to the RTC requirements of ESO.

This talk will cover the RTC problem size, the hardware solution with GPUs and the software design. Furthermore, the prototype of the hard real-time core and its performance is presented.

HARMONI's Adaptive Optics Control System: Design and Performance Update (D. Barr, Durham University)

HARMONI is a first light instrument for the European Extremely Large Telescope (ELT) that uses near-infrared integral field spectroscopy to capture detailed spectral information across astronomical objects.  The Adaptive Optics Control System (AOCS) for HARMONI is designed to support laser tomographic, single conjugate and high-contrast AO operating modes in support of the various HARMONI observing modes. We present the design and performance of the multi-AO mode AOCS for HARMONI, covering both hard and soft real time control system performance.

We provide a detailed status report of our development efforts, with a focus on the system design and the results of our recent prototyping efforts. First, we will present the prototype of the HARMONI hard real-time control system (HRTC) and its design, including timing results from a full-scale HRTC in the laboratory. We have recently updated our designs with the development of Durham's next-generation AO RTC called DAO.

We will show how DAO meets (and can exceed) the requirements HARMONI places on the AOCS to achieve its scientific goals. Finally, we present the design and status of the soft real-time control system (SRTC) for HARMONI. Including algorithm performance results that drive computation and system dimensioning and the use of the ESO RTC Toolkit.

    Session 9: VLT Projects

Entering the final design phase for the MAVIS RTC (F. Rigaut, ANU)

MAVIS (the MCAO Assisted Visible Imager & Spectrograph) will be driven by a high performance real-time control (RTC) system relying on cutting edge hardware and software technologies. To meet the extremely challenging requirements of a complex instrument like MAVIS, this forward looking implementation of the COSMIC platform is designed to support, end-to-end, a wide range of control schemes, from classical model-based approaches up to modern data-driven methodologies.

In this talk, I will review the next steps towards the final design as well as prototyping activities planned for phase C of the project.

    Session 10: Other ESO Projects

The CaNaPy RTC: towards pre-correction of the LGS beacon (D. Jenkins, ESO)

CaNaPy is an experimental LGS AO system in development at ESO in collaboration with ESA, Durham University CfAI, IAC, and INAF, with scientific contributions from Noelia Martinez at ANU.  The main goal of CaNaPy is to develop the hardware and software to realise a pre-corrected uplink LGS beam to provide better focus and stability at the 90 km sodium layer. CaNaPy will be used as a test bed to investigate the time differential technique for measuring the atmospheric TT from the LGS beacon, possible with the monostatic laser launch. The AO system of CaNaPy includes a pyramid WFS for the LGS beacon and a Shack-Hartmann WFS with a natural star used for providing the high order and low order AO measurements respectively. The CaNaPy system includes provisions for daytime AO operations, which means the RTC is required to perform the AO calculations at up to 2 kHz for this more demanding use case. The classical astronomical AO real time control concept has been adapted to deal with the unique challenges of providing an AO corrected uplink laser beam during atmospheric conditions of the day and night.

Here we present an overview of the work done on updating and adapting the astronomical AO RTC software DARC for CaNaPy and a summary of our recent (09/2023) commissioning results.

INDUSTRY KEYNOTE TALK: Server technologies in the light of accelerated computing (R. Kunz, Dell Technologies)

Dell PowerEdge server are used broadly in the industry and in scientific research to drive various workloads. Furthermore, they are the standard for ESO. In this talk we will re-introduce the server line and their capabilities, but will also discuss the need for scientific computing, like GPU acceleration, and give an outlook on some technology aspects going forward.

Day 3

INDUSTRY KEYNOTE TALK: Intelligence Processing Unit, a novel architecture providing massive parallel computing capabilities (G. Soubrane,  Graphcore)

Graphcore holds a unique position in the European AI compute ecosystem.

We will present our approach, driving unprecedented performances thanks to massive independent parallel computing capacity collocated with huge on-chip memory. We will explore how such a compute density is achieved thanks to our MIMD architecture and BSP low-precision processing, together bringing best-in-class AI compute efficiency for latency, performance, cost and energy consumption.

INDUSTRY KEYNOTE TALK: Intel technology update (F. Kuypers, Intel)

Intel has recently announced various new datacenter products and this talk only scratches the surface of what is already available or will become available in 2024. We will feature: - 4th gen Xeon, its built in accelerators and 4th gen Xeon with HBM - 5th gen Xeon and its improvements - Intel 2024 roadmap with Performance and Efficient Core products - Intel Datacenter GPU offering - Intel software tools to support all of the above

    Session 11: Machine Learning

Real-time multi-stage deep neural network control for SCExAO (B.  Pou Mulet, Barcelona Supercomputing Center (BSC)

We present a real-time implementation of a novel multi-stage deep neural network control pipeline, and validate it on the Subaru Coronagraphic Extreme Adaptive Optics (SCExAO) instrument in the Subaru Telescope. The pipeline consists of a supervised learning model based on the U-Net architecture for non-linear reconstruction of pyramid wavefront sensor images and a model-free reinforcement learning (RL) approach for predictive control. The U-Net model is trained offline with data gathered on the bench, leading to better phase reconstruction accuracy in settings with lower-modulation or higher turbulence strength. The RL model is trained online with telemetry data, allowing for non-linear predictive control that adapts to changing atmospheric conditions.

For high-framerate inference, we develop a library integrating NVIDIA’s high-performance deep learning inference framework TensorRT with the popular modular image processing library toolkit (MILK), which handles the instrument data streams. The library allows the integration of models trained offline on the bench or models trained online during observation.

Finally, we present our results on the SCExAO bench. We show how integrating TensorRT and MILK allows for high framerates necessary for on-sky deployment (1-2 KHz) and demonstrate increased performance both with the U-Net and the reinforcement learning approaches.

PO4AO: XAO control with model-based reinforcement learning (J. Nousiainen, LUT University)

Highest-contrast imaging with ELT-PCS requires a highly performant and robust control system. The main science case of nearby Exo-Earths calls for high contrast at very small angular separations of tens of milliarcseconds where the contrast is affected by quickly changing eXtreme AO residuals and quasi-static speckles. Data-driven control methods such as Reinforcement Learning (RL), a subfield of machine learning where system control is learned through interaction with the environment, hold great promise and now receive a lot of interest in the XAO field. Model-based RL provides automated, self-tuning control for AO while being efficient to execute and train. It can handle temporal and misregistration errors and adapt to non-linear wavefront sensing. In addition, the concept of RL has a huge potential for focal plane wavefront control to tackle the quasi-static speckles.

This talk will discuss recent advances, motivation, and prospects of RL methods for adaptive optics wavefront sensor control and focal plane wavefront control. I will present our RL approach called Policy Optimizations for AO (PO4AO), summarize the tests with the GHOST test bench at ESO headquarters, and discuss the prospects of running PO4AO on-sky with an optimized RTC implementation.

Performance analysis of wavefront estimation using machine learning methods for real-time control (J. Smith, ANU)

Early results of new data driven machine learning (ML) methods for wavefront estimation have shown some potentially useful improvements in simulation, however rigorous analysis of these simulated results in the astronomical domain have been difficult to evaluate due to the data driven nature of the neural networks applied. Due to their opaque operational nature, these methods require robust performance analysis before they can be considered for practical use.

We present a careful error analysis from the projection of the wavefront estimates on a set of Karhunen-Loeve modes, and compare with similar statistics generated for simulated benchmarks. We then use this technique to examine previously published estimation techniques that estimate wavefronts from Shack-Hartmann wavefront sensor images, outlining their relative strengths and weaknesses for pseudo open-loop control and point spread function reconstruction (PSF-R).

Finally, we conduct a thorough analysis of the effects of noise on conditional Generative Adversarial Network (cGAN) and UNet wavefront estimation methods and provide an assessment of suitability to control and PSF-R applications from simulated results. We find that the UNet Assisted control performance is particularly robust to photon and read-out noise when applied in simulation for low photon regimes.

    Session 12: Algorithms

Matrix-free vs. matrix-based real-time reconstruction on CPU and GPU (B. Stadler, RICAM)

In this talk, we focus on the Finite Element Wavelet Hybrid Algorithm (FEWHA), which is a real-time reconstruction algorithm that can be formulated either in a matrix-free and iterative way or as a matrix-vector-multiplication (MVM). The iterative approach has advantages such as no precomputation of the inverse and on the fly parameter updates, but does not support pipelining. We study the performance of the matrix-based vs. the matrix-free FEWHA in simulations for a MORFEO like test setting. MORFEO is an Adaptive Optics system of the ELT and fulfilling its real-time requirements is a challenging task. We will study whether the iterative approach affects the reconstruction quality in comparison to a direct solver. Moreover, we look in detail at the computational performance of the matrix-free FEWHA on a CPU vs. the matrix-based version on a GPU.

Creating and Deploying the METIS Command Matrix (H. Steuer, MPIA)

At the core of the adaptive optics control algorithms of the ELT 1st light instrument METIS is a conventional command matrix that reconstructs the wavefront in the spatial domain by translating from wavefront sensor measurements to a modal space. The method behind the creation of this command matrix is perhaps a little bit less conventional: in a first step a reconstruction matrix is created using a virtual deformable mirror allowing us to use an optimal amount of information delivered by the wavefront sensor. In a second step, a projection onto the actual control space is computed permitting for the mis-registration between sensor and actuators to be treated disjunct from the wavefront reconstruction.

This talk gives an overview of this approach with a special focus on its implementation in simulation and the transformation into an optimised design for deployment on the GPU-based METIS adaptive optics soft real-time cluster. We will present the current status of the implementation and it’s integration with the hard real time computer as well as the simulation environment used to test the implementation.

Performance study on correlation methods for the new generation of solar telescopes (N. Rodríguez, IAC)

The European Solar Telescope will have an aperture of 4.2 m, an Adaptive Secondary Mirror (ASM) and a complex Multi Conjugate Adaptive Optics (MCAO) solution to satisfy the science quality requirements. A testbench has been designed to test different AO strategies (SCAO, GLAO and MCAO) with a scaled model of the EST MCAO system. The objective of this testbench is to obtain knowledge about the different optical, wavefront reconstruction and control challenges of the MCAO problem. One key difference between night AO and solar AO is the use of correlations as a necessary process to perform the slopes computation when SH-WFS are used. Furthermore, the correlation algorithms have a known impact on the performance of the adaptive optics, as it can introduce measurement errors in the control loop.

This work analyses the computation performance of the typical correlations methods, based on distance norms (absolute difference and squared difference) or Fast Fourier Transform (FFT). It will be assed the impact of subpixel algorithms in terms of computation efficiency as well. Our study will also include a comparison of the different algorithms performance in closed-loop to develop the trade-off between computational efficiency and the resulting control quality. As a final point, we will be stating the scalability of these results to the EST case discussing possible methods to improve the overall processing load, such as parallelization.

Predictive Control Schemes for Adaptive Optics in Free Space Optical Communication(J. Torres, DLR)

Adaptive Optics (AO) is rapidly growing to be an essential part of the Free Space Optical Communications (FSOC) field. AO for FSOC poses diffeent challenges to astronomical AO due to the need to operate in worst case turbulence scenarios, at sites of potentially poor seeing conditions, and also the need to track fast moving Low Earth Orbit (LEO) satellites across the sky.

One solution to these challenges is to run a highly optimised Real-Time Control system at very high frequencies ( > 10 kHz), however this may not be necessary. At DLR-KN, we are working in collaboration with the Laboratory Charles Fabry from Institut d’Optique to develop high performance optimal controllers that have the potential to alleviate the need for such high iteration rates by modelling turbulence conditions and predicting its evolution. This model-based control can be especially fruitful in the LEO case, in which the movement of the satellite induces a high-velocity apparent motion of the high-altitude turbulence layers, leading to dominant frozen flow effects.

Here, we report on adaptations to the controllers optimised for astronomical applications to the field of FSOC and present simulation results from an end-to-end optical propagation and AO simulation, demonstrating their effectiveness.

    Session 13: Other Developments

Real-time pixel data processing from energy-resolving detectors (D. Barr, Durham University)

High-speed detectors are increasingly moving from the traditional global-shutter, frame-based readout that has been the standard in AO for the last three decades. In Durham, we are currently investigating Microwave Kinetic Inductance Detector (MKID) arrays as potential future wavefront sensors, and these introduce unique challenges for real-time control (RTC) systems.

MKIDs provide true zero-read noise, photon-counting detectors that are capable of multi-wavelength wavefront sensing, enabling novel wavefront reconstruction and control techniques. Unlike traditional CCD/CMOS sensors, each pixel in an MKID array continuously provides a real-time stream that can be processed to determine when a photon has arrived and what its energy is. The wavelength of a photon is determined by the energy response observed by the MKID, enabling the RTC to receive real-time, timestamped energy information.

In this presentation, we will discuss the challenges associated with managing the high data rates required for Extremely Large Telescope (ELT) scale Adaptive Optics (AO) systems using MKIDs. We will explore the unique demands imposed by MKID arrays and outline our approach in overcoming them. By leveraging the capabilities of DAO, we aim to effectively handle the continuous data streams, analyse the energy information in real time, and explore new methods of reconstruction using the multispectral information from the MKID.

Ethernet packet time stamping techniques for real time performance assessment (T. Grudzien & N. Benes, ESO)

ELT RTCs use an internal network based on standard ethernet. Assessing the performance of an RTC implies being able to measure when ethernet frames are sent by wavefront sensors and when frames are received or sent by the HRTC nodes. In this talk we present 3 different tools we use at ESO to timestamp ethernet traffic:

• ERSPANv2 is a feature of the recommended switches for RTCs; ERSPANv2 creates a copy of received and / or sent frames by the switch and adds a timestamp in the encapsulation header,
• The native hardware timestamping capability of certain generic-purpose network adapters,
• A PCI Express adapter from Napatech specifically designed for network capture.

We present these tools from the user perspective: the poorly documented information needed to parse the ERSPAN headers, the PTP requirements and command line options to activate hardware timestamping using tshark or tcpdump, and a few hints to get started with the Napatech.

ELT WFRTC: Software patterns and library solutions for low latency multithreading (C. Rosenquist, ESO)

The ELT Wave Front RTC (WFRTC) is the control subsystem to be used in combination with the Pre-focal Station and the Phase & Diagnostics Station to verify the ELT SCAO capabilities. Its hard real time core will be realized using HPC-class mainstream CPU servers and Ethernet networking, and dimensioned for operation at 500 Hz with sub-millisecond end-to-end latency.

In the context of the hard real time core this presentation starts with how precise CPU and memory resource locality is achieved and then continues with an overview of three recurring software patterns for inter-thread communication: queueing, scatter-gather & signals/events. Included throughout are our adopted library solutions which include both bespoke in-house and third-party C++ libraries.

Current status of the Adaptive Optics Telemetry standard (T. Gomes, Faculdade de Engenharia da Universidade do Porto)

We have designed the Adaptive Optics Telemetry (AOT) data exchange format, as a standard for sharing Adaptive Optics (AO) telemetry obtained from VIS/NIR ground-based observatories. AOT builds upon the Flexible Image Transport System (FITS), serving as a packaging step prior to sharing telemetry data (after the real-time dumping of instrument data), with the goal of providing an unambiguous and consistent access to data regardless of data source.

AOT supports diverse systems and configurations (such as different VLT/ELT-class telescopes, natural/laser guide star strategies, wavefront sensing/correcting device types) and provides flexibility in the level of detail of the data to be shared. The format was designed with two key use-cases in mind: atmospheric turbulence parameter estimation and point-spread function reconstruction, for which we provide demonstrations with real-world data (PAPYRUS@OHM and NAOMI/ERIS@VLT).

We have released the first version of the AOT specification, where we define the structure of AOT files, including detailed specifications for data fields, descriptions, data types, units, and expected dimensions. To support this format, we have made available a Python package that enables seamless data conversion, reading, writing, and exploration of AOT files. We demonstrate the flexibility of the AOT format by packaging data from five different systems, including ESO's VLT systems.

This demonstration data has been made publicly available at the ESO Archive.