rail.core.utilStages module

Stages that implement utility functions

class rail.core.utilStages.ColumnMapper(args, comm=None)

Bases: RailStage

Utility stage that remaps the names of columns.

Notes

1. This operates on pandas dataframs in parquet files.

2. In short, this does: output_data = input_data.rename(columns=self.config.columns, inplace=self.config.inplace)

config_options = {'chunk_size': 100000, 'columns': <class 'dict'>, 'inplace': False, 'output_mode': <ceci.config.StageParameter object>}

inputs = [('input', <class 'rail.core.data.PqHandle'>)]

name = 'ColumnMapper'

outputs = [('output', <class 'rail.core.data.PqHandle'>)]

run()

Run the stage and return the execution status

class rail.core.utilStages.Dereddener(args, comm=None)

Bases: RailStage

Utility stage that does dereddening

config_options = {'band_a_env': [4.81, 3.64, 2.7, 2.06, 1.58, 1.31], 'bands': 'ugrizy', 'copy_cols': [], 'dustmap_dir': <class 'str'>, 'dustmap_name': 'sfd', 'mag_name': 'mag_{band}_lsst', 'output_mode': <ceci.config.StageParameter object>}

fetch_map()

inputs = [('input', <class 'rail.core.data.Hdf5Handle'>)]

name = 'Dereddener'

outputs = [('output', <class 'rail.core.data.Hdf5Handle'>)]

run()

Run the stage and return the execution status

class rail.core.utilStages.LSSTFluxToMagConverter(args, comm=None)

Bases: RailStage

Utility stage that converts from fluxes to magnitudes

Note, this is hardwired to take parquet files as input and provide hdf5 files as output

config_options = {'bands': 'ugrizy', 'copy_cols': {}, 'flux_err_name': '{band}_gaap1p0FluxErr', 'flux_name': '{band}_gaap1p0Flux', 'mag_err_name': 'mag_err_{band}_lsst', 'mag_name': 'mag_{band}_lsst', 'mag_offset': 31.4, 'output_mode': <ceci.config.StageParameter object>}

inputs = [('input', <class 'rail.core.data.PqHandle'>)]

mag_conv = 0.9210340371976182

name = 'LSSTFluxToMagConverter'

outputs = [('output', <class 'rail.core.data.Hdf5Handle'>)]

run()

Run the stage and return the execution status

class rail.core.utilStages.RowSelector(args, comm=None)

Bases: RailStage

Utility Stage that sub-selects rows from a table by index

Notes

1. This operates on pandas dataframs in parquet files.

2. In short, this does: output_data = input_data[self.config.start:self.config.stop]

config_options = {'output_mode': <ceci.config.StageParameter object>, 'start': <class 'int'>, 'stop': <class 'int'>}

inputs = [('input', <class 'rail.core.data.PqHandle'>)]

name = 'RowSelector'

outputs = [('output', <class 'rail.core.data.PqHandle'>)]

run()

Run the stage and return the execution status

class rail.core.utilStages.TableConverter(args, comm=None)

Bases: RailStage

Utility stage that converts tables from one format to anothe

FIXME, this is hardwired to convert parquet tables to Hdf5Tables. It would be nice to have more options here.

config_options = {'output_format': <class 'str'>, 'output_mode': <ceci.config.StageParameter object>}

inputs = [('input', <class 'rail.core.data.PqHandle'>)]

name = 'TableConverter'

outputs = [('output', <class 'rail.core.data.Hdf5Handle'>)]

run()

Run the stage and return the execution status