test(reference): ground-truth tests and implementation notes (refs #87, category C1) - #117
Merged
Conversation
…#87) Category C1 of the #87 plan. 18 new cases, 229 -> 247 total. Tests and docs only, no src/ changes. This is the first part of the suite that pins actual output values. Parity compares against a vendored jsfeat that shares any inherited defect, and the invariant tests constrain answers without fixing them — #115 showed an off-by-one in a loop bound surviving both, because it changed which code path produced a pixel without changing the value for a uniform image. Two complementary kinds, in tests/reference/: Reference implementations (imgproc.test.ts). Naive, obvious versions of box_blur, sobel, scharr and the integral image, written from each operation's definition and compared on non-uniform input. All match BIT-EXACTLY, so no tolerances are involved anywhere. Closed-form values (known-values.test.ts). Where a reference implementation cannot help because the same misunderstanding would sit on both sides: the binomial kernels asserted to the bit, jsfeat's integer luma constants checked against real-valued BT.601 rather than against themselves, and inputs CONSTRUCTED from known answers — A = U diag(w) V^T requiring SVD to return the chosen singular values, and a chosen homography requiring homography2d to recover it. Three findings worth recording. Sobel and scharr use ASYMMETRIC border handling: reflect vertically (BORDER_REFLECT_101), replicate horizontally. Assuming replication in both directions makes the interior match exactly (352/352) and nearly every border pixel disagree (35/80) — a failure that reads like a wrong kernel when the kernel is fine. Now pinned by its own test. get_gaussian_kernel size 7 is [2,7,14,18,14,7,2]/64, NOT Pascal's [1,6,15,20,15,6,1]/64, though sizes 3 and 5 are the binomial rows. Flatter peak, heavier tails. Pinned so a rewrite cannot "correct" it and silently change every 7-tap blur. And a flaw in my own first draft. Re-running the #115 off-by-one against it: the ground-truth test STILL missed it. At 23x17 that mutation perturbs only radius 3, by exactly 1 — the slack a ±1 tolerance was allowing for the known truncation defect. Fixed by sweeping five shapes and by MODELLING the defect instead of tolerating it: a reference variant scales by the float reciprocal exactly as the library does, so comparison is exact at every radius, and the truncation is asserted separately as "differs from exact division only at radius 3, and only by 1". Now caught. A tolerance wide enough to absorb a known defect is wide enough to hide an unknown one. Also adds docs/implementation-notes.md, collecting the conventions and quirks established across this work: border conventions, fixed-point and rounding behaviour, the known defects (#102, #110, #111, #114) with the measurements behind them, behaviour that looks wrong but is not, data-structure gotchas, and notes on what each layer of the test suite can and cannot catch. Verified: prettier clean, tsc --noEmit clean, license-check clean, npm test 246 passed + 1 expected fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Category C1 of the #87 plan. 18 new cases, 229 → 247 total. Tests and docs only, no
src/changes.Why
This is the first part of the suite that pins actual output values. Parity compares against a vendored jsfeat that shares any inherited defect; the invariant tests constrain answers without fixing them. #115 showed an off-by-one in a loop bound surviving both, because it changed which code path produced a pixel without changing the value for a uniform image.
What is here
Reference implementations (
tests/reference/imgproc.test.ts) — naive, obvious versions ofbox_blur,sobel,scharrand the integral image, written from each operation's definition and compared on non-uniform input. All match bit-exactly, so there are no tolerances anywhere.Closed-form values (
tests/reference/known-values.test.ts) — for what a reference implementation cannot catch, since the same misunderstanding would sit on both sides. The binomial kernels asserted to the bit; jsfeat's integer luma constants checked against real-valued BT.601 rather than against themselves; and inputs constructed from known answers —A = U·diag(w)·Vᵀrequiring SVD to return the chosen singular values, a chosen homography requiringhomography2dto recover it.Three findings
1. Sobel and scharr use asymmetric border handling. Reflect vertically (
BORDER_REFLECT_101), replicate horizontally:Assuming replication in both directions makes the interior match exactly (352/352) and nearly every border pixel disagree (35/80) — which reads like a wrong kernel when the kernel is fine. Now pinned by its own test.
2.
get_gaussian_kernelsize 7 is not the binomial row. It is[2,7,14,18,14,7,2]/64, not Pascal's[1,6,15,20,15,6,1]/64— though sizes 3 and 5 are the binomial rows. Flatter peak, heavier tails. Pinned so a rewrite cannot "correct" it and silently change every 7-tap blur in the library.3. My own first draft had the flaw it was written to prevent. Re-running the #115 off-by-one against it, the ground-truth test still missed it. At 23×17 that mutation perturbs only radius 3, by exactly 1 — precisely the slack a ±1 tolerance was allowing for the known truncation defect.
Fixed two ways: sweep five shapes rather than one (8×8 at radius 1 differs in 15 pixels), and model the defect instead of tolerating it — a reference variant scales by the float reciprocal exactly as the library does, so comparison is exact at every radius, and the truncation is asserted separately as "differs from exact division only at radius 3, and only by 1". Now caught.
docs/implementation-notes.mdCollects the conventions and quirks established across this whole effort, so they are not rediscovered: border conventions, fixed-point and rounding behaviour, the known defects (#102, #110, #111, #114) with the measurements behind them, behaviour that looks wrong but is not (identity warps,
equalize_histogramon a uniform image, LK displacement limits), data-structure gotchas (8-byte padding,medianreturning the lower middle), and notes on what each layer of the suite can and cannot catch.Corrections are kept rather than edited out — including the
min(cols, rows)rule for #114 that turned out to be wrong.Remaining
C2 —
gaussian_blur(its arithmetic model still needs resolving; neither float nor a naive fixed-point reproduction matched) and the warps/resample. C3 — optional OpenCV cross-check, to be decided once C2 shows what is still unpinned.Verified: prettier clean,
tsc --noEmitclean,license-checkclean,npm test246 passed + 1 expected fail.🤖 Generated with Claude Code