ADASS XXXI

The ESO Common Pipeline Library (CPL) comprises a set of ISO-C libraries that provide a comprehensive, efficient and robust software toolkit to develop astronomical data-reduction recipes, which has been the fundamental tool for building data reduction pipelines for ESO VLT instruments since its original release in 2003, and which will continue to be the basis for ELT instrument pipelines. CPL was developed in C for reasons of efficiency and speed of execution, and because of its maturity and widespread use, it is well tested and understood. However, as the astronomical community’s preference moves more and more towards Python as the language of choice for algorithm prototyping and data reduction, there has emerged a need to provide access to CPL for users who wish to make use of the power of CPL from a Python environment.
The PyCPL project provides this access, making it possible to run existing CPL recipes from within a Python environment as well as developing new recipes in Python. These new recipes are built using the PyCPL library, a fully object-orientated, idiomatic Python interface to the powerful CPL library, while also allowing users and developers to take full advantage of the rest of the scientific Python ecosystem. In this demo we will show how both CPL and PyCPL recipes can be executed using PyEsoRex (the PyCPL equivalent of EsoRex), both on the command line and from within an interactive Python session. We will then give an example of PyCPL recipe development, demonstrating a range of PyCPL library features in the process. Finally we will highlight some of the interoperability features of PyCPL, which enable easy conversions between PyCPL objects and objects from commonly used Python libraries such as numpy, astropy and pandas.

Authors: Anthony Heng, Aidan Farrell, Anthony Horton, Nuria Lorente, Ralf Palsa, Lars Lundin, Katrina Sealey

The ESO Common Pipeline Library (CPL) comprises a set of ISO-C libraries that provide a comprehensive, efficient and robust software toolkit to develop astronomical data-reduction recipes, which has been the fundamental tool for building data reduction pipelines for ESO VLT instruments since its original release in 2003, and which will continue to be the basis for ELT instrument pipelines. CPL was developed in C for reasons of efficiency and speed of execution, and because of its maturity and widespread use, it is well tested and understood. However, as the astronomical community’s preference moves more and more towards Python as the language of choice for algorithm prototyping and data reduction, there has emerged a need to provide access to CPL for users who wish to make use of the power of CPL from a Python environment.
The PyCPL project aims to provide this access to CPL within Python. Compiled CPL structures and algorithms will be mapped to Python equivalent objects. This presents the opportunity to execute existing data reduction pipelines from within the Python environment, in addition to providing Python developers the opportunity to write their own pipelines using the existing CPL structures and algorithms, leveraging the efficiency, speed, and maturity of CPL from a pure Python ecosystem. Included with the PyCPL package will the PyEsoRex recipe execution module, which is a Python analogue of the EsoRex execution tool. PyEsoRex allows the user to run both existing C Recipes and new Python recipes and provides a standardised interface for Python recipe development.
In this talk we will introduce PyBind11: the core library that has allowed us to connect CPL to Python, and discuss how we have employed it in the project: what has worked “out of the box”, and what has needed customisation and additional C++ and Python code to produce a pyCPL which feels natural to CPL users, and attractive to those accustomed to a Python environment.