Supervisor(s): Pavel Shchekaturov (ESO, SPARTA and ELT RTC team)

Abstract: 

Neural-network controllers could improve adaptive optics (AO) correction on the VLT and the ELT, but only if their inference fits deterministically within the frame budget of the real-time computer (RTC). Average speed is not enough: a command that arrives late even once in many thousands of frames disturbs the loop. This internship provides the first measured numbers for a SPARTA-like pipeline.

The student implements a dedicated inference core that runs a small multilayer perceptron (MLP) or convolutional neural network (CNN) on one frame of wavefront-sensor data and returns the command vector. The core is a self-contained component with a fixed interface, reusable in the VLT SPARTA and ELT RTC environments through thin adapters.

The baseline implementation uses NVIDIA TensorRT on NVIDIA GPUs, with the NVIDIA RTX PRO 6000 as the baseline GPU, the L40S as an alternative and the RTX 5090 as the minimum fallback. Networks are exchanged in the open ONNX format, so other inference back-ends (for example ONNX Runtime, hand-written CUDA kernels or a portable layer such as SYCL) can be benchmarked against TensorRT behind the same interface. Vendor-specific multi-GPU features such as NVLink are not used.

The core is compared with the standard SPARTA matrix-vector multiplication (MVM) on the same hardware and the same recorded ERIS telemetry. The first evaluation measures latency and jitter (median and tail latency, deadline misses) as a function of network size, at ERIS size and at the size of ELT M4 (~5000 actuators). The study also quantifies how FP32, FP16 and INT8 precision change latency and numerical accuracy. 

Deliverables are the inference core, a reusable benchmark harness with raw timing data, and a short technical note; the results feed directly into the PhD project on a safety-supervised RL control engine. Required skills: C++, CUDA or GPU programming, basic knowledge of neural networks.