pyesorex.pyesorex

Main module for the Pyesorex PyCPL recipe execution environment.

class pyesorex.pyesorex.Pyesorex(config=None, config_format=None, **kwargs)

PyCPL recipe execution environment.

The Pyesorex class is a tool for executing CPL recipes, both classic CPL recipes written in C and PyCPL recipes written in Python.

Parameters:

  • config (str, optional) – Path of a PyEsoRex configuration file. If given Pyesorex will load it on instantiation, otherwise it will try to load a configuration from the default location, ~/.pyesorex/pyesorex.rc.

  • config_format (str, optional) – Format of the config file, either “rc” or “json” (not yet implemented). If not given Pyesorex will infer the format of the config file from the file extension.

  • display_import_errors (bool, optional) – When True any import errors encountered while searching for Python recipes will be displayed as warnings. When False import errors will be suppressed.

  • link_dir (str, optional) – The directory in which a symbolic link to each of the product files should be written. The enable/disable switch to control whether the link is actually made is the suppress_link parameter.

  • log_dir (str, optional) – Directory where to place the logfile.

  • log_file (str, optional) – Filename of logfile.

  • log_level (str, optional) – Controls the severity level of messages that will be printed to the logfile. Must be one of “debug”, “info”, “warning”, “error”, “off”.

  • msg_level (str, optional) – Controls the severity level of messages that will be printed to the terminal. Must be one of “debug”, “info”, “warning”, “error”, “off”.

  • no_datamd5 (bool, optional) – Disables the computation of the MD5 data hash for FITS product files.

  • no_checksum (bool, optional) – Disables the computation of the standard FITS product checksums

  • output_dir (str, optional) – The directory where the product files should be finally moved to (all products are first created in the current dir).

  • output_mkdir (bool, optional) – When True output and link directories will be created if they do not already exist. When False an output or link directory that does not exist will result in an error, as it does with EsoRex.

  • output_prefix (str, optional) – Prefix applied to any output file. For example, specifying ‘pre’ would translate ‘filename.fits’ to ‘pre_0000.fits’. See also the suppress_prefix parameter.

  • products_sof (str, optional) – Output file which contains the FITS files created by the recipe. If the filename ends with extension .json then a machine-readable JSON format will be used.

  • products_sof_abspath (bool, optional) – When True file paths in products SOF files will be converted to absolute paths, otherwise they will not be modified.

  • recipe_config (str, optional) – Configuration file for any selected recipe.

  • recipe_dirs (str, optional) – Directory containing recipe libraries. Note that esorex will recursively search not only the specified directory, but all sub-directories below it as well. Multiple directory heads may be specified, by separating the starting paths with colons (:).

  • suppress_link (bool, optional) – When True, no symbolic link is created to the output product. However, if False, then a symbolic link is created in the directory specified with the parameter link-dir for each product that is created by the recipe.

  • suppress_prefix (bool, optional) – When True, the original name of the output product, as produced by the recipe, is maintained. If False, then the name of the output file is changed to the “prefix_number” format. The prefix can be altered using the output_prefix option.

  • time (bool, optional) – Measure and show the recipe’s execution time.

output_dir
parameters

Parameter list containing the PyEsoRex settings.

Type:

pyesorex.parameter.ParameterList

recipe
recipe_parameters

Parameter list containing the recipe settings that will appled on the next call of run(). Will be None if no recipe is loaded, of if the currently loaded recipe has no parameters.

Type:

pyesorex.parameter.ParameterList or None

sof_location
sof
get_c_recipes()

Get a dictionary with metadata of all available classic CPL (written in C) recipes.

Calling this method triggers a recursive search for recipes in the directories set by the recipe_dirs parameter.

Returns:

A dictionary containing recipe names and RecipeMetadata objects, a NamedTuple with name, synopsis, description and version attributes.

Return type:

dict of str, RecipeMetadata

get_envars()

Get parameter values from environment variables.

Checks for environment variables with names matching the env_alias attributes of the Pyesorex parameters, and if any are found sets the value of the parameter to the value of the environment variable. This should be called after loading a config file in order to (re)apply overrides to the config file parameter values.

get_params_text(include_recipe_params=False)

Returns the current parameter values as a formatted multiline string.

By default only the PyEsoRex parameters are listed, however the parameters of the currently loaded recipe can also be included.

Parameters:

include_recipe_params (bool, optional) – If True the current parameter values for the currently loaded recipe will be included in the output. Default False.

Returns:

Multiline string listing the current parameter values.

Return type:

str

get_python_recipes(display_import_errors=False)

Get a dictionary containing all available PyCPL (written in Python) recipes.

Calling this method triggers a recursive search for recipes in the directories set by the recipe_dirs parameter.

Returns:

A dictionary containing recipe names and the corresponding cpl.ui.PyRecipe class.

Return type:

dict of str, cpl.ui.PyRecipe

get_recipe_help()

Generates the recipe help for the currently loaded recipe, based off esorex –help (recipe name)

Details recipe information including its name, usage, brief synopsis and available options

Returns:

The recipe help page for printing

Return type:

str

Raises:

RuntimeError – If no recipe is yet loaded by pyesorex, or pyesorex failed to load the recipe parameters

get_recipe_manual()

Generates the recipe manual for the currently loaded recipe, based off esorex’s –man-page.

Details recipe information including its name, description and how its used, options available for the recipe, copyright information and any contact details.

Returns:

The recipe manual page for printing

Return type:

str

Raises:

RuntimeError – If no recipe is yet loaded by pyesorex, or pyesorex failed to load the recipe parameters

get_recipes()

Get a dictionary with metadata of all available recipes.

Calling this method triggers a recursive search for recipes in the directories set by the recipe_dirs parameter.

Returns:

A dictionary containing recipe names and RecipeMetadata objects, a NamedTuple with name, synopsis, description and version attributes.

Return type:

dict of str, RecipeMetadata

get_recipes_text()

Get a summary of available recipes in text form.

Calling this method triggers a recursive search for recipes in the directories set by the recipe_dirs parameter.

Returns:

Multiline string containing a formatted, human readable summary of all available recipes. The summary lists the name and synopsis of each recipe.

Return type:

str

is_c_recipe(recipe_name)

Is there an available classic CPL (written in C) recipe with the given recipe name?

Calling this method triggers a recursive search for recipes in the directories set by the recipe_dirs parameter.

Parameters:

recipe_name (str) – Name of the recipe to check.

Returns:

True if there is a C recipe with name recipe_name.

Return type:

bool

is_python_recipe(recipe_name)

Is there an available PyCPL (written in Python) recipe with the given recipe name?

Calling this method triggers a recursive search for recipes in the directories set by the recipe_dirs parameter.

Parameters:

recipe_name (str) – Name of the recipe to check.

Returns:

True if there is a Python recipe with name recipe_name.

Return type:

bool

load_recipe(recipe_name, recipe_config=None)

Loads a recipe by name.

Parameters:

  • recipe_name (str) – Named of the recipe to load.

  • recipe_config (str, optional) – Path to recipe config file. If given Pyesorex will load the recipe config file after loading the recipe, otherwise Pyesorex will try to load a recipe config from the default location, ~/.pyesorex/<recipe_name>.rc.

See also

get_recipes

Gets a dictionary with metadata of all available recipes.

get_recipes_text

Gets a summary of available recipes in text form.

read_config(config_path=None, format=None, ignore_unsupported=False, suppress_read_error=False, recipe=False)

Read a PyEsoRex configuration file.

Reads a configuration file and sets the values of the parameters in self.parameters to the values found in the file.

It is recommended to load any required configuration file when creating the Pyesorex object by passing a config argument to the constructor. This ensures that the desired logging / console message settings are applied from the start. This method can be used to load a configuration file after instantiation, however.

Parameters:

  • config_path (str, optional) – Path to the config file. If not given this method will attempt to load a configuration file from the default location, i.e. ~/.pyesorex/pyesorex.rc.

  • format (str, optional) – Format of the config file, either “rc” or “json” (not yet implemented). If not given Pyesorex will infer the format of the config file from the file extension.

  • ignore_unsupported (bool, default=False) – If True unrecognised keys in the config file will result in a DEBUG message only, if False (default) a KeyError will be raised. Setting this to True may be useful for loading an EsoRex config file that contains settings that PyEsoRex does not yet support.

  • recipe (bool, default=False) – Set to True to load a recipe configuration file instead of a PyEsoRex configuration file. This is not inteded for use by user code, the recommended way to load a recipe configuration is to use the read_recipe_config method instead.

  • suppress_read_error (bool, default=False) – Used by read_recipe_config to suppress errors when attempting to load recipe configs from the default location, where there may or may not be a confif file. User code should not need to set this. If True a failure to open the config file will result in a DEBUG message only, if False (default) an OSError will be raised.

See also

read_recipe_config

Read a recipe configuration file.

read_recipe_config(config_path=None, format=None, ignore_unsupported=False)

Read a recipe configuration file.

Reads a configuration file and sets the values of the parameters in self.recipe_parameters to the values found in the file.

A recipe must be loaded before calling this method so that config file parser knows the names of the parameters to parse.

Parameters:

  • config_path (str, optional) – Path to the config file. If not given this method will attempt to load a configuration file from the locaation specified in the “recipe_config” parameter value, or if that is not set it will try the default location, i.e. ~/.pyesorex/<recipe_name>.rc

  • format (str, optional) – Format of the config file, either “rc” or “json” (not yet implemented). If not given Pyesorex will infer the format of the config file from the file extension.

  • ignore_unsupported (bool, default=False) – If True unrecognised keys in the config file will result in a DEBUG message only, if False (default) a KeyError will be raised. Setting this to True may be useful for loading an EsoRex config file that contains settings that PyEsoRex does not yet support.

Raises:

RuntimeError – Raised if this method is called when no recipe has been loaded, of if the recipe has no parameters attribute.

See also

read_config

Read a PyEsoRex configuration file.

run()

Run the currently loaded recipe.

Runs the currently loaded recipe using the current recipe parameter values from recipe_parameters and the input FITs files from sof.

Returns:

FrameSet object containing the details of the FITs file output by the recipe.

Return type:

cpl.ui.FrameSet

write_config(config_path=None, format=None, defaults=False, recipe=False)

Write a PyEsoRex configuration file.

Creates a new PyEsoRex configuration file containing the current values of the parameters in self.parameters.

Parameters:

  • config_path (str, optional) – Path for the config file. If not given this method will attempt to create a configuration file at the default location, i.e. ~/.pyesorex/pyesorex.rc

  • format (str, optional) – Format of the config file, either “rc” or “json” (not yet implemented). If not given Pyesorex will infer the format of the config file from the file extension.

  • defaults (bool, default=False) – Set to True to write the default values for the PyEsoRex parameters to the config file instead of the current values.

  • recipe (bool, default=False) – Set to True to create a recipe configuration file instead of a PyEsoRex configuration file. This is not inteded for use by user code, the recommended way to create a recipe configuration is to use the write_recipe_config method instead.

See also

write_recipe_config

Create a recipe configuration file.

write_recipe_config(config_path=None, format=None, defaults=False)

Write a PyEsoRex configuration file.

Creates a new recipe configuration file containing the current values of the parameters in self.recipe_parameters.

A recipe must be loaded before calling this method in order for there to be recipe parameters to write to the file.

Parameters:

  • config_path (str, optional) – Path for the config file. If not given this method will attempt to create a configuration file at the default location, i.e. ~/.pyesorex/<recipe_name>.rc

  • format (str, optional) – Format of the config file, either “rc” or “json” (not yet implemented). If not given Pyesorex will infer the format of the config file from the file extension.

  • defaults (bool, default=False) – Set to True to write the default values for the recipe parameters to the config file instead of the current values.

See also

write_config

Create a PyEsoRex configuration file.

write_sof(frames, filepath, format=None, abspaths=False)

Creates a SOF file for the given FrameSet.

Parameters:

  • frames (iterable of cpl.ui.Frame) – An iterable (e.g. a cpl.ui.FrameSet) that contains the set of Frame objects that are to be written to the SOF file.

  • filepath (pathlib.Path or str) – Path for the SOF file, either as a pathlib.Path object or a string.

  • format (str, optional) – If “json” the SOF file will be written in JSON format, if “sof” the original SOF file format will be used. If not given the format will be inferred from filepath, if it ends with .json then JSON format will be used, otherwise SOF file format.

  • abspaths (bool, optional) – If True the file paths from the Frames will be converted to absolute paths when writing them to the SOF file, otherwise they will be left unchanged. Default False.

The currently set link directory for recipe products.

This property is a pathlib.Path object corresponding to current value of the Pyesorex.properties[‘link_dir’] parameter. It can be set by assigning either a Path or a string to this property, using Pyesorex.properties[‘link_dir’].set_from_string(), the PYESOREX_LINK_DIR environment variable, the pyesorex.link-dir config file entry or the –link-dir command line option.

Returns:

Path object for the output directory.

Return type:

pathlib.Path

property output_dir

The currently set output directory for recipe products.

This property is a pathlib.Path object corresponding to current value of the Pyesorex.properties[‘output_dir’] parameter. It can be set by assigning either a Path or a string to this property, using Pyesorex.properties[‘output_dir’].set_from_string(), the PYESOREX_OUTPUT_DIR environment variable, the pyesorex.output-dir config file entry or the –output-dir command line option.

Returns:

Path object for the output directory.

Return type:

pathlib.Path

property recipe

The currently loaded recipe.

Returns:

The currently loaded recipe, or None is no recipe is loaded.

Return type:

cpl.ui.Recipe or None

property sof

The currently loaded SOF.

Returns:

A FrameSet containing the set of input files for the recipe.

Return type:

cpl.ui.FrameSet

property sof_location

The path to the SOF file.

Returns:

Path to the SOF file containing the details of the input FITs files. A None value indicates the sof was not yet set or that it was alternatively assigned via an iterable containing cpl.ui.Frame objects.

Return type:

str or None

class pyesorex.pyesorex.RecipeMetadata(name, synopsis, description, version)
description

Alias for field number 2

name

Alias for field number 0

synopsis

Alias for field number 1

version

Alias for field number 3