rail.core package

Core code for RAIL

class rail.core.DataHandle

Bases: object

Class to act as a handle for a bit of data. Associating it with a file and providing tools to read & write it to that file

__init__(tag, data=None, path=None, creator=None)

Constructor

Parameters:

  • tag (str) – The tag under which this data handle can be found in the store

  • data (DataLike | None) – The associated data

  • path (str | None) – The path to the associated file

  • creator (str | None) – The name of the stage that created this data handle

Return type:

None

close(**kwargs)

Close the associated file

Parameters:

kwargs (Any)

Return type:

None

data_size(**kwargs)

Return the size of the in memory data

Parameters:

kwargs (Any)

Return type:

int

finalize_write(**kwargs)

Finalize and close file written by chunks

Parameters:

**kwargs (Any) – Passed to call to write this chunk of data

Return type:

None

classmethod get_sub_class(class_name)

Get a particular subclass by name

Parameters:

class_name (str)

Return type:

type[DataHandle]

classmethod get_sub_classes()

Get all the subclasses

Return type:

dict[str, type[DataHandle]]

property has_data: bool

Return true if the data for this handle are loaded

property has_path: bool

Return true if the path for the associated file is defined

initialize_write(data_length, **kwargs)

Initialize file to be written by chunks

Parameters:

  • data_length (int) – Number of rows of data that we will write, used to reserve space

  • **kwargs (Any) – Information about the columns we will write

Return type:

None

interactive_type = 'Data for RAIL'
property is_written: bool

Return true if the associated file has been written

iterator(**kwargs)

Iterator over the data

Parameters:

kwargs (Any)

Return type:

Iterable

classmethod make_name(tag)

Construct and return file name for a particular data tag

Parameters:

tag (str)

Return type:

str

open(**kwargs)

Open and return the associated file

Parameters:

**kwargs (Any) – Passed to the call to open the file in question

Returns:

Newly opened file

Return type:

FileLike

Notes

This will simply open the file and return a FileLike object to the caller. It will not read or cache the data

classmethod print_sub_classes()

Print the list of all the subclasses

Return type:

None

read(force=False, **kwargs)

Read and return the data from the associated file

Parameters:

  • force (bool) – If true, force re-reading the data

  • **kwargs (Any) – Passed to the call to read the data

Returns:

Data that were read

Return type:

DataLike

Notes

This will read the entire file, and while useful for testing on small files, will not work on very large files.

set_data(data, partial=False)

Set the data for a chunk, and set the partial flag to true

Parameters:

  • data (rail.core.data.DataLike)

  • partial (bool)

Return type:

None

size(**kwargs)

Return the size of the data associated to this handle

Parameters:

kwargs (Any)

Return type:

int

suffix: str | None = ''
write(**kwargs)

Write the data to the associated file

Parameters:

kwargs (Any)

Return type:

None

write_chunk(start, end, **kwargs)

Write the data to the associated file

Parameters:

  • start (int) – Index of starting row for this chunk of data

  • end (int) – Index of ending row for this chunk of data

  • **kwargs (Any) – Passed to call to write this chunk of data

Return type:

None

class rail.core.DataStore

Bases: dict

Class to provide a transient data store

This class:

  1. associates data products with keys

  2. provides functions to read and write the various data produces to associated files

__init__(**kwargs)

Build from keywords

Note

All of the values must be data handles or this will raise a TypeError

Parameters:

kwargs (Any)

Return type:

None

add_data(key, data, handle_class, path=None, creator='DataStore')

Create a handle for some data, and insert it into the DataStore

Parameters:

  • key (str)

  • data (rail.core.data.DataLike)

  • handle_class (type[DataHandle])

  • path (str | None)

  • creator (str)

Return type:

DataHandle

add_handle(key, handle_class, path, creator='DataStore')

Create a handle for some data, and insert it into the DataStore

Parameters:

  • key (str)

  • handle_class (type[DataHandle])

  • path (str)

  • creator (str)

Return type:

DataHandle

allow_overwrite = False
open(key, mode='r', **kwargs)

Open and return the file associated to a particular key

Parameters:

  • key (str)

  • mode (str)

  • kwargs (Any)

Return type:

rail.core.data.FileLike

read(key, force=False, **kwargs)

Read the data associated to a particular key

Parameters:

  • key (str)

  • force (bool)

  • kwargs (Any)

Return type:

rail.core.data.DataLike

read_file(key, handle_class, path, creator='DataStore', **kwargs)

Create a handle, use it to read a file, and insert it into the DataStore

Parameters:

  • key (str)

  • handle_class (type[DataHandle])

  • path (str)

  • creator (str)

  • kwargs (Any)

Return type:

DataHandle

write(key, **kwargs)

Write the data associated to a particular key

Parameters:

  • key (str)

  • kwargs (Any)

Return type:

None

write_all(force=False, **kwargs)

Write all the data in this DataStore

Parameters:

  • force (bool)

  • kwargs (Any)

Return type:

None

class rail.core.FitsHandle

Bases: TableHandle

DataHandle for a table written to fits

suffix: str | None = 'fits'
class rail.core.Hdf5Handle

Bases: TableHandle

DataHandle for a table written to HDF5

interactive_type = 'dict'
suffix: str | None = 'hdf5'
class rail.core.Model

Bases: object

Class to act as wrapper for ML models

This will attach metadata to the ML model, and provide tools for validation, versioning and tracking provenance.

__init__(data, creation_class_name, version=0, catalog_tag=None, provenance=None)

Constructor

Parameters:

  • data (Any) – Model data

  • creation_class_name (str) – Name of class that created this model

  • version (int) – Version of the model

  • catalog_tag (str | None) – Associated CatalogConfigBase that defined training dataset

  • provenance (dict | None) – Provenance infomration

Return type:

None

classmethod dump(obj, path, creation_class_name='dummy', version=0, catalog_tag=None, provenance=None)

Write an object to a model file

This will promote it to a Model if it isn’t already

Parameters:

  • obj (Any) – Object to dump

  • path (str) – File to write

  • creation_class_name (str) – Name of class that created this model

  • version (int) – Version of the model

  • catalog_tag (str | None) – Associated CatalogConfigBase that defined training dataset

  • provenance (dict | None) – Provenance information

Returns:

Newly converted Model

Return type:

Model

classmethod read(path, creation_class_name='dummy', version=0, catalog_tag=None, provenance=None)

Read a model from a file.

Note that this will promote the data to a Model if it is not already

Parameters:

  • path (str) – File to read

  • creation_class_name (str) – Name of class that created this model

  • version (int) – Version of the model

  • catalog_tag (str | None) – Associated CatalogConfigBase that defined training dataset

  • provenance (dict | None) – Provenance infomration

Returns:

Newly read Model

Return type:

Model

validate(creation_class_name, version)
Parameters:

  • creation_class_name (str | None) – Name of class that created this model

  • version (int | None) – Version of the model

Return type:

None

:raises TypeError : Either creation_class_name or version does not match:

classmethod wrap(inpath, outpath, creation_class_name='dummy', version=0, catalog_tag=None, provenance=None)

Read a model from a file and write it as a Model if it is not already

Parameters:

  • inpath (str) – File to read

  • outpath (str) – File to write

  • creation_class_name (str) – Name of class that created this model

  • version (int) – Version of the model

  • catalog_tag (str | None) – Associated CatalogConfigBase that defined training dataset

  • provenance (dict | None) – Provenance information

Returns:

Newly read & converted Model

Return type:

Model

write(path)

Write a model to a file

Parameters:

path (str) – File to write

Return type:

None

class rail.core.ModelDict

Bases: dict

A specialized dict to keep track of individual estimation models objects: this is just a dict these additional features

  1. Keys are paths

  2. There is a read(path, force=False) method that reads a model object and

    inserts it into the dictionary

  3. There is a single static instance of this class

open(path, mode, **kwargs)

Open the file and return the file handle

Parameters:

  • path (str)

  • mode (str)

  • kwargs (Any)

Return type:

rail.core.data.FileLike

read(path, force=False, reader=None, **_kwargs)

Read a model into this dict

Parameters:

  • path (str)

  • force (bool)

  • reader (Callable | None)

  • _kwargs (Any)

Return type:

rail.core.data.ModelLike

write(model, path, force=False, writer=None, **_kwargs)

Write the model, this default implementation uses pickle

Parameters:

  • model (rail.core.data.ModelLike)

  • path (str)

  • force (bool)

  • writer (Callable | None)

  • _kwargs (Any)

Return type:

None

class rail.core.ModelHandle

Bases: DataHandle

DataHandle for machine learning models

interactive_type = 'numpy.ndarray'
suffix: str | None = 'pkl'
class rail.core.PointEstimationMixin

Bases: object

calculate_point_estimates(qp_dist, grid=None)

This function drives the calculation of point estimates for qp.Ensembles. It is defined here, and called from the _process_chunk method in the CatEstimator child classes.

Parameters:

  • qp_dist (Ensemble) – The qp Ensemble instance that contains posterior estimates.

  • grid (ndarray[tuple[Any, ...], dtype[_ScalarT]] | list | None) – The grid on which to evaluate the point estimate. Note that not all point estimates require a grid to be provided, by default None.

Returns:

The original qp.Ensemble with new ancillary point estimate data included. The Ensemble.ancil keys are [‘mean’, ‘mode’, ‘median’].

Return type:

qp.Ensemble

Notes

If there are particularly efficient ways to calculate point estimates for a given CatEstimator subclass, the subclass can implement any of the _calculate_<foo>_point_estimate for any of the point estimate calculator methods, i.e.:

  • _calculate_mode_point_estimate

  • _calculate_mean_point_estimate

  • _calculate_median_point_estimate

property config: StageConfig
class rail.core.PqHandle

Bases: TableHandle

DataHandle for a parquet table

interactive_type = 'pandas.core.frame.DataFrame'
suffix: str | None = 'pq'
class rail.core.QPDictHandle

Bases: DataHandle

DataHandle for dictionaries of qp ensembles

suffix: str | None = 'hdf5'
class rail.core.QPHandle

Bases: DataHandle

DataHandle for qp ensembles

interactive_type = 'qp.core.ensemble.Ensemble'
suffix: str | None = 'hdf5'
class rail.core.QPOrTableHandle

Bases: QPHandle, Hdf5Handle

DataHandle that will work with either qp.Ensembles or TableLike data

class PdfOrValue

Bases: Enum

both = 2
distribution = 0
has_dist()
Return type:

bool

has_point()
Return type:

bool

point_estimate = 1
unknown = -1
check_pdf_or_point()

Check the associated file to see if it is a QP pdf, point estimate or both

Return type:

PdfOrValue

is_qp()

Check if the associated data or file is a QP ensemble

Return type:

bool

suffix: str | None = 'hdf5'
class rail.core.RailEnv

Bases: object

Singleton class that manages introspection into the RAIL installation

This will idenfity all of the installed RAIL packages, namespaces and files, and navigate the connections between them.

This will also generate restructed text (.rst) files for the Sphinx autodoc documentation

classmethod attach_stages(to_module, silent=False)

Attach all the available stages to this module

Parameters:

  • to_module (ModuleType) – python module we are attaching stages to

  • silent (bool)

Return type:

None

Notes

This allow you to do:

from rail.stages import *
classmethod build_rail_namespace_tree()

Build a tree of the namespaces and packages in rail

Returns:

Tree of the namespaces and packages in rail

Return type:

dict[str, list[dict]]

classmethod do_api_rst(basedir='.')

Build the top-level API documentation

Parameters:

basedir (str) – Directory to write file to

Return type:

None

classmethod do_module_api_str(basedir, key, options)

Build the api rst file for a rail module

Parameters:

  • basedir (str) – Directory to write file to

  • key (str) – Name of the rail module

  • options (str) – Pre-formatted autodoc options

Return type:

None

classmethod do_namespace_api_rst(basedir, key, val)

Build the api rst file for a rail namespace

Parameters:

  • basedir (str) – Directory to write file to

  • key (str) – Name of the rail namespace

  • val (dict) – Namespace tree for the namespace

Return type:

None

classmethod do_pkg_api_rst(basedir, key, val, options, module_options)

Build the api rst file for a rail package

Parameters:

  • basedir (str) – Directory to write file to

  • key (str) – Name of the rail package

  • val (dict) – Namespace tree for the package

  • options (str) – Pre-formatted autodoc options for the package

  • module_options (str) – Pre-formatted autodoc options for modules

Return type:

None

classmethod do_stage_type_api_rst(basedir='.')

Genarate the rst files for the stage tpye documentation

Parameters:

basedir (str)

Return type:

None

classmethod import_all_packages(silent=False)

Import all the packages that are available in the RAIL ecosystem

Parameters:

silent (bool)

Return type:

None

classmethod list_rail_modules()

List all modules within rail

Returns:

Dict mapping module names to their import paths

Return type:

dict[str, str]

classmethod list_rail_namespaces()

List all the namespaces within rail

Returns:

Dict mapping the namespaces to the paths contributing to each namespace

Return type:

dict[str, list[str]]

classmethod list_rail_packages()

List all the packages that are available in the RAIL ecosystem

Returns:

Dict mapping the package names to the path to the package

Return type:

dict[str,str]

classmethod pretty_print_tree(the_dict=None, indent='')

Utility function to help print the namespace tree

This can be called recurisvely to walk the tree structure, which has nested dicts

Parameters:

  • the_dict (dict | None) – Current dictionary to print, if None it will print cls._tree

  • indent (str) – Indentation string prepended to each line

Return type:

None

classmethod print_rail_modules()

Print all the moduels that are available in the RAIL ecosystem

Return type:

None

classmethod print_rail_namespace_tree()

Print the namespace tree in a nice way

Return type:

None

classmethod print_rail_namespaces()

Print all the namespaces that are available in the RAIL ecosystem

Return type:

None

classmethod print_rail_packages()

Print all the packages that are available in the RAIL ecosystem

Return type:

None

classmethod print_rail_stage_dict()

Print an dict of all the RailSages organized by their base class

Return type:

None

class rail.core.RailPipeline

Bases: MiniPipeline

A pipeline intended for interactive use

Mainly this allows for more concise pipeline specification, along the lines of:

self.stage_1 = Stage1Class.build(…) self.stage_2 = Stage2Class.build(connections=dict(input=self.stage1.io.output), …)

And end up with a fully specified pipeline.

__init__()

Create a MiniRunner Pipeline

In addition to parent initialization parameters (see the Pipeline base class), this subclass can take these optional keywords.

Parameters:

  • callback (function(event_type: str, event_info: dict)) – A function called when jobs launch, complete, or fail, and when the pipeline aborts. Can be used for tracing execution. Default=None.

  • sleep (function(t: float)) – A function to replace time.sleep called in the pipeline to wait until the next time to check process completion Most normal usage will not need this. Default=None.

Return type:

None

static build_and_write(class_name, output_yaml, input_dict=None, stages_config=None, output_dir='.', log_dir='.', **kwargs)

Build a RailPipeline and write the config yaml for for it

Parameters:

  • class_name (str) – Full name of the class, e.g., rail.core.stage.RailPipeline

  • output_yaml (str) – Path to the output yaml file

  • input_dict (dict | None) – Dict of all the inputs needed to run the pipeline

  • stages_config (dict | None) – Stage configuration overrides

  • output_dir (str) – Directory to write pipeline outputs to

  • log_dir (str) – Directory to write pipeline log files to

  • **kwargs (Any) – Passed as arguements to the pipeline constructor

Return type:

None

classmethod get_pipeline_class(name)
Parameters:

name (str)

Return type:

type[RailPipeline]

static load_pipeline_class(class_name)

Import a particular RailPipeline subclass by name

Parameters:

class_name (str) – Full name of the class, e.g., rail.core.stage.RailPipeline

Returns:

Requested Pipeline sub-class

Return type:

type[RailPipeline]

pipeline_classes: dict[str, type[RailPipeline]] = {}
classmethod print_classes()
Return type:

None

class rail.core.RailStage

Bases: PipelineStage

Base class for rail stages

This inherits from ceci.PipelineStage and implements rail-specific data handling In particular, this provides some very useful features:

1. Access to the DataStore, which keeps track of the various data used in a pipeline, and provides access to each by a unique key.

2. Functionality to help manage multiple instances of a particular class of stage. The original ceci design didn’t have a mechanism to handle this. If you tried you would run into name clashes between the different instances. In ceci 1.7 we added functionality to ceci to allow you to have multiple instances of a single class, in particular we distinguish between the class name (cls.name) and and the name of the particular instance (self.instance_name) and added aliasing for inputs and outputs, so that different instances of PipelineStage would be able to give different names to their inputs and outputs. However, using that functionality in a consistent way requires a bit of care. So here we are providing methods to do that, and to do it in a way that uses the DataStore to keep track of the various data products.

Notes

These methods typically take a tag as input (i.e., something like “input”), but use the “aliased_tag” (i.e., something like “inform_pz_input”) when interacting with the DataStore.

In particular, the get_handle(), get_data() and input_iterator() will get the data from the DataStore under the aliased tag. E.g., if you call self.get_data(‘input’) for a Stage that has aliased “input” to “special_pz_input”, it will get the data associated to “special_pz_input” in the DataStore.

Similarly, add_handle() and set_data() will add the data to the DataStore under the aliased tag e.g., if you call self.set_data(‘input’) for a Stage that has aliased “input” to “special_pz_input”, it will store the data in the DataStore under the key “special_pz_input”.

And connect_input() will do the alias lookup both on the input and output. I.e., it is the same as calling self.set_data(inputTag, other.get_handle(outputTag, allow_missing=True), do_read=False)

__init__(args, **kwargs)

Constructor: Do RailStage specific initialization

Parameters:

  • args (Any)

  • kwargs (Any)

Return type:

None

add_data(tag, data=None)

Adds a handle to the DataStore associated to a particular tag and attaches data to it.

Parameters:

  • tag (str) – The tag (from cls.inputs or cls.outputs) for this data

  • data (rail.core.data.DataLike) – Data being added

Returns:

The data accesed by the handle assocated to the tag

Return type:

DataLike

add_handle(tag, data=None, path=None)

Adds a DataHandle associated to a particular tag

Parameters:

  • tag (str) – The tag (from cls.inputs or cls.outputs) for this data

  • data (rail.core.data.DataLike) – If not None these data will be associated to the handle

  • path (str | None) – If not None, this will be the path used to read the data

Returns:

The handle that gives access to the associated data

Return type:

DataHandle

classmethod build(**kwargs)

Return an object that can be used to build a stage

Parameters:

kwargs (Any)

Return type:

RailStageBuild

connect_input(other, inputTag=None, outputTag=None)

Connect another stage to this stage as an input

Parameters:

  • other (PipelineStage) – The stage whose output is being connected

  • inputTag (str | None) – Which input tag of this stage to connect to. None -> self.inputs[0]

  • outputTag (str | None) – Which output tag of the other stage to connect to. None -> other.outputs[0]

Returns:

The input handle for this stage

Return type:

DataHandle

entrypoint_function: str | None = None
extra_interactive_documentation: str | None = None
get_data(tag, allow_missing=True)

Gets the data associated to a particular tag

Notes

1. This gets the data via the DataHandle, and can and will read the data from disk if needed.

Parameters:

  • tag (str) – The tag (from cls.inputs or cls.outputs) for this data

  • allow_missing (bool) – If False this will raise a key error if the tag is not in the DataStore

Returns:

The data accesed by the handle assocated to the tag

Return type:

DataLike

get_handle(tag, path=None, allow_missing=False)

Gets a DataHandle associated to a particular tag

Parameters:

  • tag (str) – The tag (from cls.inputs or cls.outputs) for this data

  • path (str | None) – The path to the data, only needed if we might need to read the data

  • allow_missing (bool) – If False this will raise a key error if the tag is not in the DataStore

Returns:

The handle that give access to the associated data

Return type:

DataHandle

input_iterator(tag, **kwargs)

Iterate the input assocated to a particular tag

Parameters:

  • tag (str) – The tag (from cls.inputs or cls.outputs) for this data

  • **kwargs (Any) – These will be passed to the Handle’s iterator method

Return type:

Fixme

interactive_function: str | None = None
classmethod make_and_connect(**kwargs)

Make a stage and connects it to other stages

Notes

kwargs are used to set stage configuration, the should be key, value pairs, where the key is the parameter name and the value is value we want to assign

The ‘connections’ keyword is special, it is a dict[str, DataHandle] and should define the Input connections for this stage

Return type:

A stage

Parameters:

kwargs (Any)

name = 'RailStage'
open_model(tag='model', **kwargs)

Load the mode and/or attach it to this Stage

Parameters:

  • tag (str) – Input tag associated to the model

  • **kwargs (Any) – Should include ‘model’, see notes

Return type:

rail.core.data.ModelLike

Notes

The keyword arguement ‘model’ should be either

  1. an object with a trained model,

  2. a path pointing to a file that can be read to obtain the trained model,

  3. or a ModelHandle providing access to the trained model.

Returns:

The object encapsulating the trained model.

Return type:

Any

Parameters:

  • tag (str)

  • kwargs (Any)

set_data(tag, data, path=None, do_read=True)

Sets the data associated to a particular tag

Notes

1. If data is a DataHandle and tag is one of the input tags, then this will add an alias between the two, i.e., it will set self.config.alias[tag] = data.tag. This allows the user to make connections between stages simply by passing DataHandles between them.

Parameters:

  • tag (str) – The tag (from cls.inputs or cls.outputs) for this data

  • data (rail.core.data.DataLike) – The data being set,

  • path (str | None) – Can be used to set the path for the data

  • do_read (bool) – If True, will read the data if it is not set

Returns:

The data accessed by the handle associated to the tag

Return type:

DataLike

class rail.core.SharedParams

Bases: object

Parameters:

  • hdf5_groupname ([str] default=photometry) – name of hdf5 group for data, if None, then set to ‘’

  • chunk_size ([int] default=10000) – Number of objects per chunk for parallel processing or to evalute per loop in single node processing

  • zmin ([float] default=0.0) – The minimum redshift of the z grid or sample

  • zmax ([float] default=3.0) – The maximum redshift of the z grid or sample

  • nzbins ([int] default=301) – The number of gridpoints in the z grid

  • dz ([float] default=0.01) – delta z in grid

  • nondetect_val ([float] default=99.0) – value to be replaced with magnitude limit for non detects

  • nonobserved_val ([float] default=-99.0) – guard value for non-observations

  • bands ([list] default=['mag_u_lsst', 'mag_g_lsst', 'mag_r_lsst', 'mag_i_lsst', 'mag_z_lsst', 'mag_y_lsst']) – Names of columns for magnitude by filter band

  • err_bands ([list] default=['mag_err_u_lsst', 'mag_err_g_lsst', 'mag_err_r_lsst', 'mag_err_i_lsst', 'mag_err_z_lsst', 'mag_err_y_lsst']) – Names of columns for magnitude errors by filter band

  • err_dict ([dict] default={'mag_u_lsst': 'mag_err_u_lsst', 'mag_g_lsst': 'mag_err_g_lsst', 'mag_r_lsst': 'mag_err_r_lsst', 'mag_i_lsst': 'mag_err_i_lsst', 'mag_z_lsst': 'mag_err_z_lsst', 'mag_y_lsst': 'mag_err_y_lsst', 'redshift': None}) – dictionary that contains the columns that will be used topredict as the keys and the errors associated with that column as the values.If a column does not havea an associated error its value shoule be None

  • mag_limits ([dict] default={'mag_u_lsst': 27.79, 'mag_g_lsst': 29.04, 'mag_r_lsst': 29.06, 'mag_i_lsst': 28.62, 'mag_z_lsst': 27.98, 'mag_y_lsst': 27.05}) – Limiting magnitudes by filter

  • band_a_env ([dict] default={'mag_u_lsst': 4.81, 'mag_g_lsst': 3.64, 'mag_r_lsst': 2.7, 'mag_i_lsst': 2.06, 'mag_z_lsst': 1.58, 'mag_y_lsst': 1.31}) – Reddening parameters

  • ref_band ([str] default=mag_i_lsst) – band to use in addition to colors

  • redshift_col ([str] default=redshift) – name of redshift column

  • id_col ([str] default=object_id) – name of the object ID column

  • object_id_col ([str] default=objectId) – name of object id column

  • zp_errors ([list] default=[0.1, 0.1, 0.1, 0.1, 0.1, 0.1]) – BPZ adds these values in quadrature to the photometric errors

  • calc_summary_stats ([bool] default=False) – Compute summary statistics

  • calculated_point_estimates ([list] default=[]) – List of strings defining which point estimates to automatically calculate using qp.Ensemble.Options include, ‘mean’, ‘mode’, ‘median’.

  • recompute_point_estimates ([bool] default=False) – Force recomputation of point estimates

  • replace_error_vals ([list] default=[0.1, 0.1, 0.1, 0.1, 0.1, 0.1]) – list of values to replace negative and nan mag err values

  • filter_list ([list] default=['DC2LSST_u', 'DC2LSST_g', 'DC2LSST_r', 'DC2LSST_i', 'DC2LSST_z', 'DC2LSST_y']) – list of filter files names (with no ‘.sed’ suffix). Filters must bein FILTER dir. MUST BE IN SAME ORDER as ‘bands’

  • leaf_size ([int] default=15) – The leaf size for tree algorithms.

  • max_wavelength ([float] default=12000) – The maximum rest-frame wavelength

  • min_wavelength ([float] default=250) – The minimum rest-frame wavelength.

  • redshift_key ([str] default=redshifts) – The keyword of the redshift group in the hdf5 dataset.

static copy_param(param_name)

Return a copy of one of the shared parameters

Parameters:

param_name (str) – Name of the parameter to copy

Returns:

Copied parameter

Return type:

Param

static set_param_default(param_name, default_value)

Change the default value of one of the shared parameters

Parameters:

  • param_name (str) – Name of the parameter to copy

  • default_value (Any) – New default value

Return type:

None

static set_param_defaults(**kwargs)

Change the default value of several of the shared parameters

Parameters:

**kwargs (Any) – Key, value pairs of parameter names and default values

Return type:

None

class rail.core.StageIO

Bases: object

A small utility class for Stage Input/ Output

This make it possible to get access to stage inputs and outputs as attributes rather that by using the get_handle() method.

In short it maps

a_stage.get_handle(‘input’, allow_missing=True) to a_stage.input

This allows users to be more concise when writing pipelines.

__init__(parent)
Parameters:

parent (RailStage)

class rail.core.TableHandle

Bases: DataHandle

DataHandle for single tables of data

interactive_type = 'A tablesio-compatible table'
set_data(data, partial=False)

Set the data for a chunk, and set the partial flag if this is not all the data

Parameters:

  • data (rail.core.data.TableLike)

  • partial (bool)

Return type:

None

suffix: str | None = None
rail.core.copy_param(param_name)

Return a copy of one of the shared parameters

Parameters:

param_name (str) – Name of the parameter to copy

Returns:

Copied parameter

Return type:

Param

rail.core.set_param_default(param_name, default_value)

Change the default value of one of the shared parameters

Parameters:

  • param_name (str) – Name of the parameter to copy

  • default_value (Any) – New default value

Return type:

None

rail.core.set_param_defaults(**kwargs)

Change the default value of several of the shared parameters

Parameters:

**kwargs (Any) – Key, value pairs of parameter names and default values

Return type:

None

rail.core