feat(indexing): advise payload splitting and soft-commit per call - #196
Open
adityamparikh wants to merge 3 commits into
Open
feat(indexing): advise payload splitting and soft-commit per call#196adityamparikh wants to merge 3 commits into
adityamparikh wants to merge 3 commits into
Conversation
Inline indexing payloads are generated by the model as tool arguments, so a single oversized call is truncated at the model's output limit and fails to parse. Tell the model, once in the server instructions and briefly in each inline indexing tool description, to split inputs larger than a few hundred KB across several calls. The guidance is a size, not a document count, because record sizes vary widely. Splitting multiplies commits, so indexDocuments now ends with a soft commit (waitFlush=false, waitSearcher=true, softCommit=true) instead of a hard commit: documents are searchable when the tool returns, while the segment fsync is left to Solr's autoCommit (15 s in the _default configset). The README documents the trade-off for operators whose configset disables autoCommit. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019zJ9WA8nNyA5Yxb8ueM7vV Signed-off-by: Aditya Parikh <aditya.m.parikh@gmail.com>
Replace the concatenated string descriptions on the four inline indexing tools with text blocks, matching the style SearchService already uses. The payload-splitting sentence is now spelled out in each block instead of being appended from a constant; McpToolRegistrationTest still guards it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019zJ9WA8nNyA5Yxb8ueM7vV Signed-off-by: Aditya Parikh <aditya.m.parikh@gmail.com>
The splitting clause was repeated in all four tool descriptions and in the server instructions, contradicting the rule the new instructions test states. The tool descriptions revert to main; the instructions carry the guidance, and the per-tool registration test goes with it. The instructions test now asserts the actual clause instead of any sentence containing 'commit'. The unit test's when(commit(...)).thenReturn(null) stubs were no-ops and are dropped. README keeps one sentence on the soft commit. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiUHyyXLTo9ATdgg8eRFZJ Signed-off-by: Aditya Parikh <aditya.m.parikh@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two small changes to the inline indexing tools (
index-json-documents,index-csv-documents,index-xml-documents,index-markdown-documents).indexDocumentsnow ends with a soft commit (waitFlush=false, waitSearcher=true, softCommit=true) instead of a hard commit. Documents are still searchable when the tool returns; the segment fsync is left to Solr'sautoCommit, which the_defaultconfigset enables at 15 s. The transaction log covers durability in between. README notes that a configset withautoCommitdisabled should enable it.Tests
IndexingServiceTestverifies the soft-commit overload is used and the hard-commit overload is not.McpToolRegistrationTestfails if anyindex-*tool description drops the splitting guidance.McpServerInstructionsTestguards the instructions text inapplication.properties.Verification
./gradlew buildon Java 25: 407 tests, 0 failures, 7 skipped. Spotless applied.Notes
index-documentstool with aformatargument.🤖 Generated with Claude Code
https://claude.ai/code/session_019zJ9WA8nNyA5Yxb8ueM7vV