fix(local): compare score_threshold in the query's own score direction (#1370) - #1371
fix(local): compare score_threshold in the query's own score direction (#1370)#1371Anai-Guo wants to merge 1 commit into
Conversation
Recommend (best_score/sum_scores), discovery and context queries score through a sigmoid, so bigger is always better for them regardless of the collection's distance. The sort order already special-cased these query types on top of distance_to_order(), but the score_threshold comparison a few lines below branched on required_order alone. On a Euclidean or Manhattan collection it therefore cut with the operator meant for raw distances and dropped every point that should have passed. Hoist the decision into a single bigger_is_better flag used by both the sort and the threshold check, so the two cannot drift apart again.
✅ Deploy Preview for poetic-froyo-8baba7 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review. 📝 WalkthroughWalkthrough
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to This localized change aligns score-threshold filtering with the query’s score direction and fixes empty results for affected local Recommend, Discover, and Context queries; no actionable merge-blocking risk remains after normal checks and review. Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Fixes #1370.
Problem
In local/in-memory mode,
score_thresholdis compared in the wrong direction for Recommend (best_score/sum_scores), Discover and Context queries whenever the collection's distance is Euclid or Manhattan. The threshold silently drops every point that should have passed, soquery_pointsreturns an empty result set.LocalCollection.search()decides the sort order like this:The
isinstance(...)arm is there becausecalculate_recommend_best_scores,calculate_recommend_sum_scores,calculate_discovery_scoresandcalculate_context_scoresall build their score on top ofcalculate_distance_core, which already negates Euclid/Manhattan, and then run it throughscaled_fast_sigmoid/fast_sigmoid. For these query types bigger is always better, whatever the collection's distance is.The threshold check a few lines below did not repeat that override:
On a Euclid/Manhattan collection
required_orderisSMALLER_IS_BETTER, so the points — already sorted biggest-first — are cut withscore > score_threshold, and the very first point breaks the loop.Fix
Hoist the decision into a single
bigger_is_betterflag and use it for both the sort order and the threshold comparison, so the two cannot drift apart again. The flag is the exact expression the sort branch already used, so ordering behaviour is unchanged for every query type (includingNaiveFeedbackQuery, which keeps followingrequired_orderas before).Reproduction
Verification
Run against
dev, forCosine/Euclid/Manhattan:best_score, threshold below every score[]0.45,0.0,-0.2)score >= thresholdexactly[]smaller is better)score <= thresholdruff-format --line-length=99(v0.4.3, per.pre-commit-config.yaml) reports the file already formatted.tests/test_in_memory.pyandtests/conversionspass; the 4 failures intests/test_local_persistence.pyare a pre-existing Windows tempfile-locking artifact and reproduce identically on an unmodified tree.🤖 Generated with Claude Code