YOLO-E for Open Vocabulary Object Detection - #19
Draft
mrpositron wants to merge 5 commits into
Draft
mrpositron wants to merge 5 commits into
mrpositron wants to merge 5 commits into
Conversation
Adapt the YOLOE plugin to the new TableParameter and let the user choose what YOLOE's output is stored as. Prompt-capable YOLOE checkpoints are all open vocabulary `-seg` variants that produce masks and boxes together, so the new `instance_segmentation` tick selects which of the two is written, not what the model computes: - unticked: `object_detection` annotations, today's behaviour. - ticked: `segmentation_mask` annotations carrying the RLE mask and its bounding box on the same record, so no detail is lost. Inference runs with `retina_masks=True` so masks come back at the source resolution and need no resizing. Classes move from a comma-separated string to a TableParameter with one class per row, read defensively since rows are not validated before they reach the operator. Multi-word class names are supported: `set_classes` only rejects " " itself, which it uses as a background sentinel. Also register the plugin in plugins.toml, where it was missing, add a README, and pin the CLIP text encoder fork that text prompting needs so it is not pip-installed at runtime. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Rename the `classes` table parameter to `prompts` and give it the same two
columns SAM3 uses: a required `prompt` and an optional `label` that falls back
to the prompt when left empty. Rows sharing a label now merge into a single
annotation class, so `car` and `truck` can both map to `vehicle`.
Blank and whitespace-only rows are rejected instead of being silently skipped,
as are duplicate prompts — two rows with the same prompt and different labels
are ambiguous, since only one can win the class index. `TableParameter` only
validates cell types and column names, so the operator enforces this itself.
Build the {category_id: label_id} map by reading `model.names` back after
`set_classes` and mapping it through the row labels, rather than by row
position: `set_classes` keeps the checkpoint's own ordering when the requested
prompts are a permutation of its existing names, which would otherwise
mislabel every class.
Bump the lightly_studio floor to the version that introduces TableParameter.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The branch had replaced main's Python .gitignore with a different 25-line version, since this branch diverged before main's landed. Main's version already covers everything the replacement did apart from test databases and local Claude settings, so restore it rather than carrying a competing copy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Merge the overlapping open-vocabulary bullets, fold the instance_segmentation sub-bullets into one line, and drop two notes already covered elsewhere: the default collection name is in the annotation_source parameter row, and retina_masks is an internal detail with no user-facing knob. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What has changed and why?
(Delete this: Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.)
How has it been tested?
(Delete this: Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration.)
Did you update Readme.md and plugins.toml?