Q-space Lanczos is corrupted by NumPy 1.26.4 built on Python 3.14
Summary
QSpaceLanczos.run_FT can silently corrupt its Krylov vectors when tdscha is
run with NumPy 1.26.4 built from source on Python 3.14. A nominally
out-of-place masked product can reuse the input storage and mutate the vector:
before = L_q.copy()
weighted = L_q * mask_dot
np.shares_memory(L_q, weighted) # True in the failing environment
np.max(np.abs(L_q - before)) # 5.8e-8, not zero
The first Hermitian Lanczos step then has b != c, and subsequent
coefficients grow exponentially. The resulting spectrum is wrong without an
exception or warning.
This should be fixed independently on main, even though a defensive
workaround currently exists on the interpolation development branch.
Affected environment
The failure has been reproduced with:
- Python 3.14.5
- NumPy 1.26.4, built locally from the PyPI source distribution
- tdscha q-space Lanczos through JuliaCall
It has not been reproduced with the same code, input, Python interpreter,
and Julia backend after placing NumPy 2.4.6 first on PYTHONPATH.
The evidence does not establish that every NumPy version below 2 is affected.
NumPy 1.26 predates Python 3.14, so the immediately actionable compatibility
bug is that tdscha currently permits this unsupported combination:
requires-python = ">=3.8"
dependencies = ["numpy", ...]
The build requirement is similarly only numpy>=1.20.0.
Reproducer and invariant
The quickest end-to-end regression test is only three Lanczos steps on a
small q-space test ensemble, with neither ensemble reinitialization nor
reorthogonalization:
lanczos = QSpaceLanczos(ensemble, use_wigner=True, lo_to_split=None)
lanczos.init()
lanczos.prepare_raman(unpolarized=0)
lanczos.run_FT(3, verbose=False, reorthogonalize=False)
np.testing.assert_allclose(lanczos.b_coeffs, lanczos.c_coeffs,
rtol=1e-11, atol=1e-18)
assert np.all(np.isfinite(lanczos.a_coeffs))
assert np.all(np.isfinite(lanczos.b_coeffs))
The test should also exercise the exact masked metric operation and assert
that its right-hand operand is unchanged. The existing small data under
tests/test_qspace should make this test fast and portable.
Proposed fix
I suggest addressing both the immediate unsafe operation and the unsupported
dependency resolution:
-
Keep masked metric products explicitly out-of-place:
weighted_right = np.empty_like(right)
np.multiply(right, mask_dot, out=weighted_right)
value = np.vdot(left, weighted_right)
-
Add Python-version-aware NumPy lower bounds so that pip cannot resolve
Python 3.14 to NumPy 1.26. The exact marker should follow the oldest NumPy
release officially supporting each Python version.
-
Add a CI job for the newest supported Python/NumPy pair and the three-step
Hermiticity test above.
A runtime error for a known unsupported Python/NumPy pair would also be safer
than allowing a calculation to complete with plausible but incorrect data.
Q-space Lanczos is corrupted by NumPy 1.26.4 built on Python 3.14
Summary
QSpaceLanczos.run_FTcan silently corrupt its Krylov vectors when tdscha isrun with NumPy 1.26.4 built from source on Python 3.14. A nominally
out-of-place masked product can reuse the input storage and mutate the vector:
The first Hermitian Lanczos step then has
b != c, and subsequentcoefficients grow exponentially. The resulting spectrum is wrong without an
exception or warning.
This should be fixed independently on
main, even though a defensiveworkaround currently exists on the interpolation development branch.
Affected environment
The failure has been reproduced with:
It has not been reproduced with the same code, input, Python interpreter,
and Julia backend after placing NumPy 2.4.6 first on
PYTHONPATH.The evidence does not establish that every NumPy version below 2 is affected.
NumPy 1.26 predates Python 3.14, so the immediately actionable compatibility
bug is that tdscha currently permits this unsupported combination:
The build requirement is similarly only
numpy>=1.20.0.Reproducer and invariant
The quickest end-to-end regression test is only three Lanczos steps on a
small q-space test ensemble, with neither ensemble reinitialization nor
reorthogonalization:
The test should also exercise the exact masked metric operation and assert
that its right-hand operand is unchanged. The existing small data under
tests/test_qspaceshould make this test fast and portable.Proposed fix
I suggest addressing both the immediate unsafe operation and the unsupported
dependency resolution:
Keep masked metric products explicitly out-of-place:
Add Python-version-aware NumPy lower bounds so that pip cannot resolve
Python 3.14 to NumPy 1.26. The exact marker should follow the oldest NumPy
release officially supporting each Python version.
Add a CI job for the newest supported Python/NumPy pair and the three-step
Hermiticity test above.
A runtime error for a known unsupported Python/NumPy pair would also be safer
than allowing a calculation to complete with plausible but incorrect data.