fix(audio): keep the selected command on voice/STT-dictated messages - #2992
Open
RoyBA wants to merge 8 commits into
Open
fix(audio): keep the selected command on voice/STT-dictated messages#2992RoyBA wants to merge 8 commits into
RoyBA wants to merge 8 commits into
Conversation
Auto-attach the UI-selected command to the user message produced during an audio turn, so voice-dictated messages carry the command just like typed ones. The command is sent with audio_start, stored on the session for the turn, cleared on audio_end, and attached in Message.__post_init__. Co-Authored-By: GitHub Copilot <noreply@github.com>
RoyBA
requested review from
asvishnyakov,
hayescode and
sandangel
as code owners
July 28, 2026 20:16
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 7 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Message.from_dict deserializes incoming client messages and resumed thread history. Because the audio-command fallback lives in the shared __post_init__, a command-less typed message (or resumed step) created during an active audio turn would wrongly inherit the turn's command. Reset the command from the payload in from_dict so deserialized messages stay authoritative; app-constructed transcription messages still inherit as intended. Addresses PR review feedback. Co-Authored-By: GitHub Copilot <noreply@github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Carries selected commands through voice/STT turns so dictated messages match typed-message behavior.
Changes:
- Sends the selected command with
audio_start. - Stores and attaches the command to generated user messages.
- Adds command inheritance and deserialization tests.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
libs/react-client/src/useChatInteract.ts |
Emits command with audio start. |
libs/react-client/src/useAudio.ts |
Forwards command when starting audio. |
frontend/src/components/chat/MessageComposer/VoiceButton.tsx |
Reads and passes the selected command. |
backend/chainlit/socket.py |
Manages command state across audio turns. |
backend/chainlit/session.py |
Adds session command state. |
backend/chainlit/message.py |
Attaches commands to generated user messages. |
backend/tests/test_message.py |
Tests command attachment behavior. |
Suppressed comments (1)
backend/chainlit/socket.py:492
- This unconditional clear races with a subsequent audio turn.
endConversationmarks the clientoffbefore emittingaudio_end, while this handler awaits the app'son_audio_end; a quick newaudio_startcan therefore overwritecurrent_command, after which the older handler both exposes the new command to the old callback and clears it here. Associate the command with a turn/context (or serialize audio turns) so overlapping handlers cannot misattribute or erase commands.
session.current_command = None
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
A declined on_audio_start (or disabled audio) previously left the UI-selected command on the session. Since the frontend sends no audio_end for a rejected start, the turn cleanup never ran and a later server-created user message could inherit the stale command. Record the command only once the connection is accepted. Addresses PR review feedback. Co-Authored-By: GitHub Copilot <noreply@github.com>
Collaborator
|
@codex review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a command is selected in the composer (e.g.
/search) and the user dictates their message with the voice/STT feature instead of typing it, the message is sent without the selected command. Typing the exact same message correctly includes the command.Root cause
The selected command lives in frontend state (
persistentCommandState) and is attached to the message only in the text-composer submit path (theclient_messageevent). The audio path (audio_start→audio_chunk→audio_end) never transmits the command, and the transcribed user message is created on the backend (in the app'son_audio_endhandler), so it has no knowledge of the UI selection.Fix
Carry the selected command through the audio turn and auto-attach it to the user message produced during that turn.
Frontend — send the selected command with
audio_start:VoiceButtonreadspersistentCommandStateand passesselectedCommand?.idtostartConversation.useAudio.startConversation(command?)→useChatInteract.startAudioStream(command?)→socket.emit('audio_start', { command }).Backend — store it for the turn and attach it:
BaseSessiongains acurrent_commandfield.audio_startstoressession.current_command;audio_endclears it, so it only applies to that audio turn.Message.__post_init__auto-attachescurrent_commandtouser_messages that don't already carry a command.Backward compatibility
Fully backward compatible. The
audio_startpayload is optional (old clients that emit no payload keep working), the new frontend arguments are optional, and the existingMessage.commandfield is reused. Typed messages and assistant messages are unchanged.Testing
backend/tests/test_message.py: auto-attach onuser_message, explicit command takes precedence, no-op when no command is set, and assistant messages never inherit the command.mypy,ruff,pnpm type-check, ESLint, and Prettier all pass.Out of scope
modeshas the same gap (also only attached in the text submit path). Left for a follow-up to keep this PR focused on commands.Summary by cubic
Voice/STT-dictated messages now keep the selected command like typed messages; previously they dropped it.
audio_start; the payload is optional, so old clients keep working.audio_end.Message.__post_init__auto-attaches the session command to command-lessuser_messages;from_dictresets from the payload so typed or resumed messages never inherit an active audio turn's command.Written for commit 93d7102. Summary will update on new commits.