Skip to content

Add ParametricInverter, which fits a spectral line profile to every pixel - #23

Open
roytsmart wants to merge 7 commits into
mainfrom
feature/parametric-inverter
Open

Add ParametricInverter, which fits a spectral line profile to every pixel#23
roytsmart wants to merge 7 commits into
mainfrom
feature/parametric-inverter

Conversation

@roytsmart

Copy link
Copy Markdown
Contributor

MartInverter solves for the radiance in every voxel of the scene, which is underdetermined: the tutorial configuration has 81,920 unknowns and only 32,768 measurements. ParametricInverter instead solves for a handful of parameters in every spatial pixel, which makes the problem overdetermined (12,288 unknowns for a three-parameter model) and lets the Doppler shift be measured to a small fraction of a velocity bin.

The parameters of neighboring pixels are not independent, since each sensor pixel collects light from many spatial pixels, so the fit is a single optimization over every parameter of every pixel simultaneously. This is done with the Adam optimizer on a GPU.

What's here

  • ctis.Regridder assembles the sparse weights of a linear instrument into a single torch CSR matrix, so the forward model is differentiable and the adjoint comes from automatic differentiation. CSR is used instead of scatter-add (index_add_) because it is both ~2.5x faster and, unlike scatter-add, bitwise deterministic on CUDA devices. Determinism matters for the conjugate-gradient solvers planned next.

    Note this adjoint is the exact transpose of the forward model, which backproject is notbackproject applies an extra normalization to conserve flux.

  • AbstractLinearInstrument.response exposes the two diagonal factors of the noiseless forward model, so an external differentiable implementation can reproduce image() exactly without reimplementing it. Both IdealInstrument and OptikaInstrument implement it.

  • GaussianModel evaluates a Gaussian line profile, integrated analytically across each velocity bin. Bin integration is essential rather than cosmetic: the reconstruction grid is usually comparable to or coarser than the width of the line, so sampling at bin centers would make a sub-bin Doppler shift nearly unobservable.

    The free width parameter is the nonthermal width, with the thermal and instrumental widths held fixed. This makes the lower bound on the observed width exact rather than a penalty, and means the physically interesting quantity is the one which receives an uncertainty, instead of being recovered afterwards from a difference of comparable squares.

Verification

  • test_forward_matches_instrument asserts the torch forward model reproduces instrument.image() for both IdealInstrument and OptikaInstrument (agreement to ~2e-7). This is the test that guards the whole approach.
  • test__call__recovery builds a scene the model can represent exactly, forward-models it, inverts, and asserts every parameter is recovered with a correlation above 0.95. In practice the fit reaches r = 0.98 (intensity), 0.99 (velocity), 0.99 (width); with Poisson noise, 0.90 / 0.97 / 0.88.
  • The Regridder is checked against regridding.regrid_from_weights, for the adjoint identity, for bitwise determinism, and with torch.autograd.gradcheck.

Notes for review

  • torch is an optional dependency (pip install ctis[torch]), imported lazily. It is added to the test and doc extras so the new tests actually run in CI.
  • response is a new abstract member of AbstractLinearInstrument, so any subclass outside this repo would need to implement it.
  • Two things a caller has to get right, both called out in the docstrings: the sensor must be large enough to hold the dispersed scene (voxels that fall off the edge are unconstrained), and the velocity bins should be comparable to the line width.
  • No docs page yet — only the jupyter-execute examples in the docstrings.

Next step is to swap Adam for Levenberg-Marquardt, which converges in fewer operator applications and yields per-pixel error bars from JᵀWJ.

🤖 Generated with Claude Code

https://claude.ai/code/session_01B6wnJue6RxMmKBn5sD8gDL

… pixel

`MartInverter` solves for the radiance in every voxel of the scene, which is
underdetermined: the tutorial configuration has 81,920 unknowns and only 32,768
measurements. `ParametricInverter` instead solves for a handful of parameters in
every *spatial* pixel, which makes the problem overdetermined (12,288 unknowns
for a three-parameter model) and lets the Doppler shift be measured to a small
fraction of a velocity bin.

The parameters of neighboring pixels are not independent, since each sensor
pixel collects light from many spatial pixels, so the fit is a single
optimization over every parameter of every pixel simultaneously. This is done
with the Adam optimizer on a GPU.

Adds three pieces:

* `ctis.Regridder` assembles the sparse `weights` of a linear instrument into a
  single `torch` CSR matrix, so the forward model is differentiable and the
  adjoint comes from automatic differentiation. CSR is used instead of
  scatter-add because it is both faster and, unlike scatter-add, bitwise
  deterministic on CUDA devices.

* `AbstractLinearInstrument.response` exposes the two diagonal factors of the
  noiseless forward model, so that an external differentiable implementation can
  reproduce `image()` exactly without reimplementing it. Both `IdealInstrument`
  and `OptikaInstrument` implement it, and a test asserts the `torch` forward
  model matches `image()` for each.

* `GaussianModel` evaluates a Gaussian line profile, integrated analytically
  across each velocity bin. Bin integration is essential rather than cosmetic,
  since the reconstruction grid is usually comparable to or coarser than the
  width of the line. The free width parameter is the *nonthermal* width, with
  the thermal and instrumental widths held fixed, so the physically interesting
  quantity is the one which receives an uncertainty.

`torch` is an optional dependency, installed with `pip install ctis[torch]`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6wnJue6RxMmKBn5sD8gDL
@codecov

codecov Bot commented Aug 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (d9a30b7) to head (c593898).

Additional details and impacted files
@@            Coverage Diff             @@
##              main       #23    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files           23        30     +7     
  Lines          861      1658   +797     
==========================================
+ Hits           861      1658   +797     
Flag Coverage Δ
unittests 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

roytsmart and others added 6 commits August 2, 2026 17:45
The Read the Docs build failed because `plt.colorbar` cannot hold an
`astropy` quantity in its norm: `quantity_support` covers the axes but not the
colorbar. Strip the unit from the color values and move it into the label.

The `Regridder` example built its values on the CPU while the operator defaults
to a CUDA device when one is available, so it raised a device mismatch on any
machine with a GPU. This did not show up on Read the Docs, which has no GPU.
Create the values on `regridder.device` instead.

Also fixes `ParametricInverter` rejecting images which carry an uncertainty,
which is the documented way to use the instrument's own noise model:
`MartInverter` cannot consume an `UncertainScalarArray`, so the uncertainty is
now dropped before computing the initial guess.

Adds tests for the three previously uncovered branches: weights which carry a
unit, an explicitly supplied uncertainty, and an uncertainty attached to the
images.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6wnJue6RxMmKBn5sD8gDL
Mirrors the structure of the MART tutorial and uses the same synthetic scene,
so the two inversion approaches can be compared side by side.

Beyond reconstructing the scene, the notebook plots the fitted intensity,
Doppler velocity, and nonthermal width maps, which a voxel-based inversion can
only produce by post-processing the reconstructed cube, and compares the fitted
velocity against the intensity-weighted centroid of the true scene.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6wnJue6RxMmKBn5sD8gDL
The notebook used a narrower velocity grid than the MART tutorial, which
clipped the wings of the test pattern: `ctis.scenes.gaussians` places
components at +/-200 km/s with a width of 30 km/s. It now uses the same
velocity grid, rest wavelength, scene and sensor grids, plate scale,
dispersion, and dispersion angles, so the two inversions can be compared
directly.

The MART tutorial also adds a background equal to 1 percent of the maximum of
the scene. That background is flat in wavelength, so a single Gaussian cannot
represent it, and the velocity fitted in the faint pixels it dominates is not
meaningful. The notebook now says so, and masks the velocity comparison to the
brightest pixels using the same total-radiance threshold that `plot_moments`
applies through `percentile_radiance`. The fitted velocity agrees with the
median velocity of the true scene with a Pearson's r of 0.82 in those pixels.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6wnJue6RxMmKBn5sD8gDL
The natural units of a backprojection differ between instruments:
`IdealInstrument` returns an energy radiance, while `OptikaInstrument` returns
a photon radiance. `ParametricInverter` starts its fit from the moments of a
short MART reconstruction and converted that cube into the unit the fit works
in, which raised a `UnitConversionError` for the latter, since a photon
radiance cannot be converted into an energy radiance without the energy per
photon.

`backproject` already accepts a `unit` argument for exactly this reason, but
`MartInverter` did not expose it. Add a `unit` field to `MartInverter` which is
forwarded to `backproject`, and have `ParametricInverter` request the unit its
model works in. The iteration is unaffected, since the multiplicative
correction is a ratio of two backprojections and is therefore dimensionless.

The existing tests missed this because the only tests which ran the whole
inversion used `IdealInstrument`; the `OptikaInstrument` test exercised only
the forward model. Adds a test which runs the whole inversion against an
`OptikaInstrument`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6wnJue6RxMmKBn5sD8gDL
The `guess` argument of `ParametricInverter` accepted only a reconstructed
scene, which was then reduced to its moments. It now also accepts a dictionary
of physical parameters, named according to the `parameters` of the model, which
is used directly.

Since `ParametricInversionResult.parameters` is a dictionary of exactly this
form, the result of one fit can warm-start another:

    result = inverter(images)
    result = inverter(images, guess=result.parameters)

which is useful for continuing a fit that ran out of iterations, for a raster
where each exposure starts from the previous one, or for supplying a velocity
field which is already known.

Each value may be a full map over the spatial axes of the scene, or a scalar
which is broadcast over them.

To support this, the unit of each physical parameter is now declared by the
spectral model through the new `AbstractSpectralModel.unit` method, rather than
being hardcoded by the inverter, and the inverter exposes it as
`ParametricInverter.unit_parameters`. This also removes the assumption that
every model has exactly the three parameters of `GaussianModel`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6wnJue6RxMmKBn5sD8gDL
The merit function assumed the measurement was shot-noise limited, with an
arbitrary floor of one electron squared. A real sensor has a floor set by its
read noise, which for ESIS is four electrons, so the variance was underestimated
by a factor of sixteen. Because only a small fraction of the detector is lit by
the scene, the merit function was then almost entirely read noise: on an ESIS
observation it started at 9.58 and could not be moved below 9.57 no matter how
many iterations were taken. Faint pixels, which are the ones dominated by read
noise, were also weighted far too heavily against bright ones.

The instrument's own noise model is now evaluated once at the starting guess and
held fixed. Evaluating it once means the weights do not follow the model, which
would bias the fitted radiance. On the same ESIS observation the merit function
now starts at 1.00, as it should for a fit which reaches the noise floor, and
the Doppler velocity of the brightest pixels is recovered with a correlation of
0.74 rather than 0.52.

A measurement whose variance is zero now receives zero weight rather than an
infinite one, so `variance_min` is no longer needed to avoid a division by zero
and defaults to zero.

With the noise model corrected it becomes clear that the fit overfits: in the
lit region of that observation it reproduces the measurement better than the
true scene does, because a CTIS measures only a few projections and three
parameters per spatial pixel leaves the fit only marginally overdetermined.
Adds an optional `regularization` weight, off by default, which penalizes the
mean squared first difference of the physical parameters. The penalty is
applied to the physical parameters rather than the unconstrained ones, since a
large change in an unconstrained parameter is a small change in a velocity near
its bound.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6wnJue6RxMmKBn5sD8gDL
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant