Computing Hyperbolic Magnitudes
Last successfully run: Feb 9, 2025
Implementation of Lupton et al. (1999) by Jan Luca van den Busch.
Hyperbolic magnitudes aim to overcome limitations of classical magnitudes, which are logarithmic in flux. Hyperbolic magnitudues are implemented using the inverse hyperbolic sine and therefore have a linear behaviour in flux at low signal to noise, which gradually transitions to the classical logarithmic scaling at high signal to noise (i.e. equivalent to classical magnitudes in this limit).
This notebooks provides an example of how to convert classical to
hyperbolical magnitudes using the interactive versions of pipeline
stages HyperbolicSmoothing and HyperbolicMagnitudes.
If you’re interested in running this in pipeline mode, see
03_Hyperbolic_Magnitude.ipynb
in the pipeline_examples/core_examples/ folder.
import matplotlib.pyplot as plt
import rail.interactive as ri
import tables_io
from rail.utils.path_utils import find_rail_file
Install FSPS with the following commands:
pip uninstall fsps
git clone --recursive https://github.com/dfm/python-fsps.git
cd python-fsps
python -m pip install .
export SPS_HOME=$(pwd)/src/fsps/libfsps
LEPHAREDIR is being set to the default cache directory:
/home/runner/.cache/lephare/data
More than 1Gb may be written there.
LEPHAREWORK is being set to the default cache directory:
/home/runner/.cache/lephare/work
Default work cache is already linked.
This is linked to the run directory:
/home/runner/.cache/lephare/runs/20260601T134116
A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.2.6 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.
If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.
Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/ipykernel_launcher.py", line 18, in <module>
app.launch_new_instance()
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/traitlets/config/application.py", line 1082, in launch_instance
app.start()
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/ipykernel/kernelapp.py", line 758, in start
self.io_loop.start()
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/tornado/platform/asyncio.py", line 211, in start
self.asyncio_loop.run_forever()
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
self._run_once()
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/asyncio/base_events.py", line 1909, in _run_once
handle._run()
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/ipykernel/utils.py", line 71, in preserve_context
return await f(*args, **kwargs)
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 621, in shell_main
await self.dispatch_shell(msg, subshell_id=subshell_id)
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 478, in dispatch_shell
await result
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 372, in execute_request
await super().execute_request(stream, ident, parent)
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 834, in execute_request
reply_content = await reply_content
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 464, in do_execute
res = shell.run_cell(
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/ipykernel/zmqshell.py", line 663, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3077, in run_cell
result = self._run_cell(
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3132, in _run_cell
result = runner(coro)
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/IPython/core/async_helpers.py", line 128, in _pseudo_sync_runner
coro.send(None)
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3336, in run_cell_async
has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3519, in run_ast_nodes
if await self.run_code(code, result, async_=asy):
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3579, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_4090/2416626524.py", line 2, in <module>
import rail.interactive as ri
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/rail/interactive/__init__.py", line 3, in <module>
from . import calib, creation, estimation, evaluation, tools
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/rail/interactive/calib/__init__.py", line 3, in <module>
from rail.utils.interactive.initialize_utils import _initialize_interactive_module
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/rail/utils/interactive/initialize_utils.py", line 17, in <module>
from rail.utils.interactive.base_utils import (
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/rail/utils/interactive/base_utils.py", line 10, in <module>
rail.stages.import_and_attach_all(silent=True)
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/rail/stages/__init__.py", line 74, in import_and_attach_all
RailEnv.import_all_packages(silent=silent)
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/rail/core/introspection.py", line 541, in import_all_packages
_imported_module = importlib.import_module(pkg)
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/rail/som/__init__.py", line 1, in <module>
from rail.creation.degraders.specz_som import *
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/rail/creation/degraders/specz_som.py", line 15, in <module>
from somoclu import Somoclu
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/somoclu/__init__.py", line 11, in <module>
from .train import Somoclu
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/somoclu/train.py", line 25, in <module>
from .somoclu_wrap import train as wrap_train
File "/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/somoclu/somoclu_wrap.py", line 11, in <module>
import _somoclu_wrap
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
File /opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/numpy/core/_multiarray_umath.py:44, in __getattr__(attr_name)
39 # Also print the message (with traceback). This is because old versions
40 # of NumPy unfortunately set up the import to replace (and hide) the
41 # error. The traceback shouldn't be needed, but e.g. pytest plugins
42 # seem to swallow it and we should be failing anyway...
43 sys.stderr.write(msg + tb_msg)
---> 44 raise ImportError(msg)
46 ret = getattr(_multiarray_umath, attr_name, None)
47 if ret is None:
ImportError:
A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.2.6 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.
If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.
Warning: the binary library cannot be imported. You cannot train maps, but you can load and analyze ones that you have already saved.
The problem occurs because either compilation failed when you installed Somoclu or a path is missing from the dependencies when you are trying to import it. Please refer to the documentation to see your options.
Next we load some DC2 sample data that provides LSST ugrizy magnitudes and magnitude errors, which we want to convert to hyperbolic magnitudes.
testFile = find_rail_file("examples_data/testdata/test_dc2_training_9816.pq")
test_mags = tables_io.read(testFile)
column_list None
Determining the smoothing parameters
First we run the HyperbolicSmoothing stage. This stage computes the
smoothing parameter (called \(b\) in Lupton et al. 1999), which
determines the transition between the linear and logarithmic behaviour
of the hyperbolic magnitudes.
The input for this stage is a table containing magnitudes and
magnitude errors per object (fluxes are also supported as input data by
setting is_flux=True in the configuration). In this example, we
assume that the magnitude zeropoint is 0.0 and that we want to convert
all 6 LSST bands. This can be specified with the value_columns and
error_columns parameters, which list the names of the magnitude
columns and their corresponding magnitude errors.
lsst_bands = "ugrizy"
configuration = dict(
value_columns=[f"mag_{band}_lsst" for band in lsst_bands],
error_columns=[f"mag_err_{band}_lsst" for band in lsst_bands],
zeropoints=[0.0] * len(lsst_bands),
is_flux=False,
)
smooth_params = ri.tools.photometry_tools.hyperbolic_smoothing(
data=test_mags, **configuration
)
Inserting handle into data store. input: None, HyperbolicSmoothing
Inserting handle into data store. parameters: inprogress_parameters.pq, HyperbolicSmoothing
/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/hyperbolic/magnitudes.py:285: FutureWarning: The provided callable <function nanmedian at 0x7f532946a200> is currently using DataFrameGroupBy.median. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "median" instead.
stats = df.groupby(Keys.field).agg(np.nanmedian)
/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/hyperbolic/magnitudes.py:285: FutureWarning: The provided callable <function nanmedian at 0x7f532946a200> is currently using DataFrameGroupBy.median. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "median" instead.
stats = df.groupby(Keys.field).agg(np.nanmedian)
/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/hyperbolic/magnitudes.py:285: FutureWarning: The provided callable <function nanmedian at 0x7f532946a200> is currently using DataFrameGroupBy.median. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "median" instead.
stats = df.groupby(Keys.field).agg(np.nanmedian)
/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/hyperbolic/magnitudes.py:285: FutureWarning: The provided callable <function nanmedian at 0x7f532946a200> is currently using DataFrameGroupBy.median. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "median" instead.
stats = df.groupby(Keys.field).agg(np.nanmedian)
/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/hyperbolic/magnitudes.py:285: FutureWarning: The provided callable <function nanmedian at 0x7f532946a200> is currently using DataFrameGroupBy.median. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "median" instead.
stats = df.groupby(Keys.field).agg(np.nanmedian)
/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/hyperbolic/magnitudes.py:285: FutureWarning: The provided callable <function nanmedian at 0x7f532946a200> is currently using DataFrameGroupBy.median. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "median" instead.
stats = df.groupby(Keys.field).agg(np.nanmedian)
The output of this stage is a table of relevant statistics required
to compute the hyperbolic magnitudes per filter: - the median flux error
- the zeropoint (which can be computed by comparing fluxes and
magnitudes in the original hyperbolic code) - the reference flux
\(f_{\rm ref}\) that corresponds to the given zeropoint - the
smoothing parameter \(b\) (in terms of the absolute and the relative
flux \(x = f / f_{\rm ref}\)
The field ID column is currently not used by the RAIL module and can
be ignored.
smooth_params["parameters"]
| flux error | zeropoint | ref. flux | b relative | b absolute | ||
|---|---|---|---|---|---|---|
| filter | field ID | |||||
| mag_u_lsst | 0 | 1.559839e-11 | 0.0 | 1.0 | 1.625332e-11 | 1.625332e-11 |
| mag_g_lsst | 0 | 3.286980e-12 | 0.0 | 1.0 | 3.424989e-12 | 3.424989e-12 |
| mag_r_lsst | 0 | 3.052049e-12 | 0.0 | 1.0 | 3.180194e-12 | 3.180194e-12 |
| mag_i_lsst | 0 | 4.441195e-12 | 0.0 | 1.0 | 4.627666e-12 | 4.627666e-12 |
| mag_z_lsst | 0 | 7.823318e-12 | 0.0 | 1.0 | 8.151793e-12 | 8.151793e-12 |
| mag_y_lsst | 0 | 1.785106e-11 | 0.0 | 1.0 | 1.860057e-11 | 1.860057e-11 |
Computing the magnitudes
Based on the smoothing parameters, the hyperbolic magnitudes are
computed with be computed by HyperbolicMagnitudes.
The input for this module is, again, the table with magnitudes and
magnitude errors and the output table of HyperbolicSmoothing.
test_hypmags = ri.tools.photometry_tools.hyperbolic_magnitudes(
data=test_mags,
parameters=smooth_params["parameters"],
**configuration,
)
Inserting handle into data store. input: None, HyperbolicMagnitudes
Inserting handle into data store. parameters: flux error zeropoint ref. flux b relative filter field ID
mag_u_lsst 0 1.559839e-11 0.0 1.0 1.625332e-11
mag_g_lsst 0 3.286980e-12 0.0 1.0 3.424989e-12
mag_r_lsst 0 3.052049e-12 0.0 1.0 3.180194e-12
mag_i_lsst 0 4.441195e-12 0.0 1.0 4.627666e-12
mag_z_lsst 0 7.823318e-12 0.0 1.0 8.151793e-12
mag_y_lsst 0 1.785106e-11 0.0 1.0 1.860057e-11
b absolute
filter field ID
mag_u_lsst 0 1.625332e-11
mag_g_lsst 0 3.424989e-12
mag_r_lsst 0 3.180194e-12
mag_i_lsst 0 4.627666e-12
mag_z_lsst 0 8.151793e-12
mag_y_lsst 0 1.860057e-11 , HyperbolicMagnitudes
Inserting handle into data store. output: inprogress_output.pq, HyperbolicMagnitudes
/opt/hostedtoolcache/Python/3.10.20/x64/lib/python3.10/site-packages/rail/tools/photometry_tools.py:319: FutureWarning: The provided callable <function nanmedian at 0x7f532946a200> is currently using SeriesGroupBy.median. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "median" instead.
.agg(np.nanmedian)
The output of this module is a table with hyperbolic magnitudes and their corresponding error.
Note: The current default is to relabel the columns names by
substituting mag_ by mag_hyp_. If this substitution is not
possible, the column names are identical to the input table with
classical magnitudes.
test_hypmags["output"]
| mag_hyp_u_lsst | mag_hyp_err_u_lsst | mag_hyp_g_lsst | mag_hyp_err_g_lsst | mag_hyp_r_lsst | mag_hyp_err_r_lsst | mag_hyp_i_lsst | mag_hyp_err_i_lsst | mag_hyp_z_lsst | mag_hyp_err_z_lsst | mag_hyp_y_lsst | mag_hyp_err_y_lsst | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 18.040370 | 0.005046 | 16.960892 | 0.005001 | 16.653413 | 0.005001 | 16.506310 | 0.005001 | 16.466378 | 0.005001 | 16.423906 | 0.005003 |
| 1 | 21.615533 | 0.009551 | 20.709402 | 0.005084 | 20.533851 | 0.005048 | 20.437566 | 0.005075 | 20.408885 | 0.005193 | 20.388203 | 0.005804 |
| 2 | 21.851866 | 0.011146 | 20.437067 | 0.005057 | 19.709715 | 0.005015 | 19.312630 | 0.005016 | 18.953412 | 0.005023 | 18.770441 | 0.005063 |
| 3 | 19.976499 | 0.005477 | 19.128676 | 0.005011 | 18.803485 | 0.005005 | 18.619996 | 0.005007 | 18.546590 | 0.005014 | 18.479452 | 0.005041 |
| 4 | 22.294717 | 0.015481 | 21.242782 | 0.005182 | 20.911803 | 0.005084 | 20.731707 | 0.005118 | 20.700288 | 0.005308 | 20.644994 | 0.006211 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 10220 | 25.732646 | 0.301680 | 25.301790 | 0.047027 | 25.099622 | 0.036055 | 25.180361 | 0.055825 | 25.295404 | 0.108750 | 25.229366 | 0.226270 |
| 10221 | 25.251545 | 0.205102 | 24.512358 | 0.023323 | 24.345662 | 0.018623 | 24.434138 | 0.028559 | 24.547622 | 0.055349 | 24.678486 | 0.140864 |
| 10222 | 25.147493 | 0.187751 | 24.113802 | 0.016640 | 23.828346 | 0.012276 | 23.711119 | 0.015380 | 23.755514 | 0.027202 | 23.830545 | 0.065739 |
| 10223 | 26.305978 | 0.435503 | 25.067304 | 0.038089 | 24.770026 | 0.026890 | 24.586800 | 0.032711 | 24.781555 | 0.068406 | 24.653411 | 0.137773 |
| 10224 | 26.429216 | 0.461142 | 25.548904 | 0.058784 | 24.983338 | 0.032494 | 24.889564 | 0.042924 | 24.836702 | 0.071907 | 24.752944 | 0.150422 |
10225 rows × 12 columns
This plot shows the difference between the classical and hyperbolic magnitude as function of the classical \(r\)-band magnitude. The turn-off point is determined by the value for \(b\) estimated above.
filt = "r"
mag_class = test_mags[f"mag_{filt}_lsst"]
magerr_class = test_mags[f"mag_err_{filt}_lsst"]
mag_hyp = test_hypmags["output"][f"mag_hyp_{filt}_lsst"]
magerr_hyp = test_hypmags["output"][f"mag_hyp_err_{filt}_lsst"]
fig = plt.figure(dpi=100)
plt.axhline(y=0.0, color="k", lw=0.55)
plt.scatter(mag_class, mag_class - mag_hyp, s=1)
plt.xlabel("Classical magnitudue")
plt.ylabel("Classical $-$ hyperbolic magnitude")
plt.title("$r$-band magnitude")
Text(0.5, 1.0, '$r$-band magnitude')
