feat(abot): add multi-session ABot-World serving and observability - #31
Open
youngmagician114514 wants to merge 8 commits into
Open
feat(abot): add multi-session ABot-World serving and observability#31youngmagician114514 wants to merge 8 commits into
youngmagician114514 wants to merge 8 commits into
Conversation
Handle minimal CPU pipeline stubs without requiring CUDA Graph methods. Validate CUDA Graph environment values explicitly and document the attention and CUDA Graph opt-in controls. Verification: - 138 focused ABot serving, scheduler, NCCL, and trace tests passed - branch Ruff and whitespace checks passed - four-GPU trace runner syntax check passed
Apply Ruff formatting and import ordering to the ABot serving, NCCL, scheduler, and validation changes. Verification: - Ruff check and format check passed - git diff --check passed
Use tracked workload assets, keep upstream trace regeneration optional, and register the missing service example.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds ABot-World multi-session interactive serving with bounded process and
process-NCCL workers, deadline-aware continuation batching, opt-in CUDA Graph
execution, and reproducible serving observability tooling.
Motivation
ABot-World previously lacked an end-to-end runtime for multiple concurrent
interactive sessions. This PR adds explicit session ownership, lifecycle
cleanup, capacity enforcement, frame-credit-aware EDF scheduling, and raw
runtime facts for reproducible evaluation.
Type of Change
Changes Made
processandprocess-nccl.Testing
Test commands:
Focused result:
138 passed.Checklist
ruff)pre-commit run --all-files)pytest tests/)[Feature] Brief descriptionRelated Issues
None.
Additional Notes
private production trace.
results/,.orig, archives, and raw experiment files are excluded.GPU Architecture Support
Performance Impact
The batching and CUDA Graph paths are opt-in and retain eager fallback.
Batch-time priors are profile-specific; this PR does not claim a universal
throughput improvement.