fix: support mixed-length batches for models with fixed padding (#703) - #704

Open

mohmedmm wants to merge 5 commits into

qdrant:mainfrom

mohmedmm:fix-fixed-padding-ragged-batches

Commits on Sep 5, 2026

Fix padding configuration in preprocessor_utils.py
mohmedmm
authored
6f35a0a
Add test for loading tokenizer with pad_to_multiple_of
mohmedmm
authored
e6db57a
Improve tokenizer padding handling
mohmedmm
authored
498681b
Improve tokenizer test cases and assertions
mohmedmm
authored
6fb5e84
fix: address code review feedback on padding handling and coverage
mohmedmm
committed
6bb4944