Skip to content

Error with numpy <= 2 #31

Description

@mesonepigreco

Q-space Lanczos is corrupted by NumPy 1.26.4 built on Python 3.14

Summary

QSpaceLanczos.run_FT can silently corrupt its Krylov vectors when tdscha is
run with NumPy 1.26.4 built from source on Python 3.14. A nominally
out-of-place masked product can reuse the input storage and mutate the vector:

before = L_q.copy()
weighted = L_q * mask_dot

np.shares_memory(L_q, weighted)       # True in the failing environment
np.max(np.abs(L_q - before))          # 5.8e-8, not zero

The first Hermitian Lanczos step then has b != c, and subsequent
coefficients grow exponentially. The resulting spectrum is wrong without an
exception or warning.

This should be fixed independently on main, even though a defensive
workaround currently exists on the interpolation development branch.

Affected environment

The failure has been reproduced with:

  • Python 3.14.5
  • NumPy 1.26.4, built locally from the PyPI source distribution
  • tdscha q-space Lanczos through JuliaCall

It has not been reproduced with the same code, input, Python interpreter,
and Julia backend after placing NumPy 2.4.6 first on PYTHONPATH.

The evidence does not establish that every NumPy version below 2 is affected.
NumPy 1.26 predates Python 3.14, so the immediately actionable compatibility
bug is that tdscha currently permits this unsupported combination:

requires-python = ">=3.8"
dependencies = ["numpy", ...]

The build requirement is similarly only numpy>=1.20.0.

Reproducer and invariant

The quickest end-to-end regression test is only three Lanczos steps on a
small q-space test ensemble, with neither ensemble reinitialization nor
reorthogonalization:

lanczos = QSpaceLanczos(ensemble, use_wigner=True, lo_to_split=None)
lanczos.init()
lanczos.prepare_raman(unpolarized=0)
lanczos.run_FT(3, verbose=False, reorthogonalize=False)

np.testing.assert_allclose(lanczos.b_coeffs, lanczos.c_coeffs,
                           rtol=1e-11, atol=1e-18)
assert np.all(np.isfinite(lanczos.a_coeffs))
assert np.all(np.isfinite(lanczos.b_coeffs))

The test should also exercise the exact masked metric operation and assert
that its right-hand operand is unchanged. The existing small data under
tests/test_qspace should make this test fast and portable.

Proposed fix

I suggest addressing both the immediate unsafe operation and the unsupported
dependency resolution:

  1. Keep masked metric products explicitly out-of-place:

    weighted_right = np.empty_like(right)
    np.multiply(right, mask_dot, out=weighted_right)
    value = np.vdot(left, weighted_right)
  2. Add Python-version-aware NumPy lower bounds so that pip cannot resolve
    Python 3.14 to NumPy 1.26. The exact marker should follow the oldest NumPy
    release officially supporting each Python version.

  3. Add a CI job for the newest supported Python/NumPy pair and the three-step
    Hermiticity test above.

A runtime error for a known unsupported Python/NumPy pair would also be safer
than allowing a calculation to complete with plausible but incorrect data.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions