Archiv Detail

Vortragende Person/Vortragende Personen:
Dr. Jakob Nordin

Diese Veranstaltung ist Teil der Veranstaltungsreihe „NFDITalks“.

Astronomical observatories today produce high-throughput alert streams based not only on photons (light) hitting earth, but also cosmic rays, neutrinos and gravitational waves. Together, these real-time data streams tell a story of a dynamic universe expanding from the Big Bang and filled with violent events beyond energy scales reachable at earth. However, effectively using these data sources requires tools for managing information flows: computations need to be fast and scalable and uphold FAIR principles, while still allowing for scientific creativity.

We here present AMPEL as our approach to these challenges. AMPEL is a modular, scalable, cross-platform framework with explicit provenance tracking, suited for systematically processing large, complex, heterogenous datasets in real-time or not. This includes analyzing, selecting, combining, updating, enriching and reacting to data. Although primarily developed to solve challenges in multi-messenger astrophysics, AMPEL is general enough to be used in other fields where reproducibility and flexibility are simultaneously required.

AMPEL is written in Python, enables users to build analyses out of hierarchies of single-purpose units, and coordinates the execution of these units on a stream of data. AMPEL can execute multiple independent analyses at once, de-duplicating calculations requested by multiple users and recording the provenance of derived data. Analyses are described in a static, YAML-based configuration language, and can be developed and tested locally before being transferred to a cluster for large-scale execution.

Access via Zoom or YouTube