Add unit tests for VLLMModel core methods (#724) - #1377
Open
sneha4175 wants to merge 1 commit into
Open
Conversation
Adds CPU-runnable unit tests for the pure-Python logic of VLLMModel, as requested in huggingface#724, covering the vllm boundary and loglikelihood math: - _generate: greedy scoring SamplingParams, generation-path field mapping, logprobs toggling, and the temperature=0 multi-sample guard - _create_auto_model: model-args wiring, optional quantization/load_format, subfolder revision handling, and data-parallel deferral - _loglikelihood_tokens: continuation logprob slicing, summation and argmax - max_length / add_special_tokens / tokenizer properties - build_vllm_token_prompts batching vllm is mocked via a lightweight fake package injected into sys.modules with a recording SamplingParams stand-in, so the tests run without a GPU.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds unit tests for the
VLLMModelclass methods, as requested by @lewtun in #724 (unit tests to "catch subtle bugs like [#721] and also protect us a bit from silent, yet breaking changes on the vllm side").The existing
tests/unit/models/vllm/test_vllm_model.pyonly exercised tokenizer/chat-template wiring. This PR adds coverage for the pure-Python logic of the model, focusing on the boundary with vllm'sSamplingParams(exactly the area #721 broke) and the loglikelihood math.Methods now covered
_generate— the vllm boundary:generate=False):temperature=0,prompt_logprobs=1,max_tokens=1,detokenize=False, and prompts wrapped viabuild_vllm_token_promptsn,max_tokens,stop, andlogprobsmapping (incl.returns_logitson/off)num_samples > 1+temperature == 0guard raisingValueError, and that sampling is allowed with a positive temperature_create_auto_model— model-args wiring (max_model_len,seedcasting,enforce_eager, tensor-parallel size), optionalquantization/load_formatforwarding,subfolder→revisionhandling, and data-parallel deferral (returnsNone, sets the ray executor backend)_loglikelihood_tokens— continuation logprob slicing, summation, and rank-1 argmax bookkeeping, plus the resultingModelResponsefieldsmax_length,add_special_tokens,tokenizerpropertiesbuild_vllm_token_promptsbatching/empty-input behaviourHow vllm is mocked
vllm is a heavy GPU dependency and
VLLMModelis gated behind@requires("vllm")(it degrades to a placeholder when vllm is absent). To keep these tests CPU-runnable while still exercising the model's own code, avllm_model_modulefixture injects a lightweight fakevllm(andray) package intosys.modules— including a recordingSamplingParamsstand-in — patchesis_package_availableand reloads the module so the real class loads, then reloads again on teardown to restore the original import state. Instances are built with__new__to skip__init__(no real engine), setting only the attributes each method needs. When vllm is installed (thedev-gpuCI env), the fakes simply shadow it for the duration of each test.Test / lint results
pytest tests/unit/models/vllm/test_vllm_model.py -k "Properties or GenerateSamplingParams or CreateAutoModel or LoglikelihoodTokens or TokenPromptsBatching"→ 14 passedruff checkandruff format --check→ cleanCloses #724 (or fixes part of it — happy to adjust scope).
Before submitting
VLLMModelbase methods #724