| Reproducibility, a core principle of science, is close to impossible to achieve in its strict sense in time-domain astronomy - we cannot redo an observation of a transient event once it is gone. Related features such as provenance and analysis reproducibility is further challenged by fractured software development, heterogeneous data sources and an emphasis on real-time decisions. The AMPEL data processing framework has been designed to allow processing of high throughput heterogeneous data streams while guaranteeing that all decisions are traced and every analysis can be reproduced. AMPEL is built on three main concepts that will be introduced here: a modular structure which introduces code-to-data in time-domain science, a flexible NoSQL database backend and a formalism for specifying/defining an entire real-time analysis in a compact and sharable format ("recipes"). The structure of AMPEL furthermore encourages shared software development and makes it easy to make data and pipelines open source. |
