We present a real-time implementation of a novel multi-stage deep neural network control pipeline, and validate it on the Subaru Coronagraphic Extreme Adaptive Optics (SCExAO) instrument in the Subaru Telescope. The pipeline consists of a supervised learning model based on the U-Net architecture for non-linear reconstruction of pyramid wavefront sensor images and a model-free reinforcement learning (RL) approach for predictive control. The U-Net model is trained offline with data gathered on the bench, leading to better phase reconstruction accuracy in settings with lower-modulation or higher turbulence strength. The RL model is trained online with telemetry data, allowing for non-linear predictive control that adapts to changing atmospheric conditions.

For high-framerate inference, we develop a library integrating NVIDIA’s high-performance deep learning inference framework TensorRT with the popular modular image processing library toolkit (MILK), which handles the instrument data streams. The library allows the integration of models trained offline on the bench or models trained online during observation.

Finally, we present our results on the SCExAO bench. We show how integrating TensorRT and MILK allows for high framerates necessary for on-sky deployment (1-2 KHz) and demonstrate increased performance both with the U-Net and the reinforcement learning approaches.

Bartomeu Pou Mulet, Barcelona Supercomputing Center (BSC)