feat(audio): expose streaming dictation and owned live calls

[lidge-jun]

Summary

• Expose streaming dictation at WS /v1/audio/transcriptions/stream and external GPT-Live through the existing Live/Realtime paths. Browser clients use an audio-specific encoded proxy-key carrier; upstream account credentials remain server-owned.
• Bind external call aliases to their creating key/provider/account, preserve native callers, bound connection lifetime and pending frames, and propagate cancellation through credential acquisition and socket cleanup. Connectivity-only completion remains neutral for inference account health.
• Ordinary stack: feat(audio): expose standalone transcription API #4391 (codex/audio-transcription -> dev) -> this PR (codex/audio-streaming) -> feat(dashboard): add dictation and live voice API controls #4395 (codex/audio-connections). Review this layer against its parent; leave the chain open.

Verification

• Added protocol, carrier, expiry/capacity, native-platform-key, Pool ownership, failure-event and shutdown regression tests. NOT RUN locally, per owner instruction; remote CI is the executable gate.
• Source review identified and drove corrections for admission, credential replacement, cancellation ownership and protocol/accounting boundaries. Independent inherited closure reviews: PASS, with no remaining actionable finding in their bounded scope.
• Exact head 011f2dff5ca3667b88f090fe711c4b2c77efd190, parent 71e22d967ffafbd3492935c299623c9127eaf17b: remote run 34687731903 passed audio-client 11, transcription 24, dictation/lifecycle 11 and call-bindings 4 cases, plus typecheck/privacy and 1,979 dashboard tests. Existing journal restore failures in test 2/4 and macOS 2/2 remain separate per explicit owner scope. This is not a whole-run-green claim; PR remains draft.
• git diff --check passed for patch whitespace only. It is not a product test.
• Protocol contracts were checked against the pinned local Codex source and installed desktop bundle. No microphone, personal audio or actual provider call was used for this layer.

Checklist

• Scope stays focused and avoids unrelated cleanup.
• Docs or release notes were updated when needed.
• Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/audio-streaming

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

✅ Deterministic PR hygiene checks passed.

[lidge-jun]

리뷰 · 우선순위 57 / 80

이 PR은 오디오 스택 wp2입니다. 부모는 draft #4391(codex/audio-transcription)이고, 여기서는 WS /v1/audio/transcriptions/stream 받아쓰기 스트림과 외부 GPT-Live/Realtime 호출 소유권을 붙입니다. 지금 dev HEAD 20861aebf에는 /v1/audio/* 자체가 없고, 기존 src/server/live.ts 쪽은 네이티브/내부 호출 중심입니다. 이 브랜치는 src/server/audio-dictation.ts, audio-live.ts, audio-client.ts, live-call-bindings.ts를 추가하고 src/server/index.ts·ws-bridge.ts에 라우팅을 연결합니다. 브라우저 WebSocket은 opencodex-audio + opencodex-key. base64url 키 캐리어만 허용하고, 업스트림 ChatGPT/OpenAI 자격증명은 서버가 고릅니다. Live 호출은 rtc_ocx_ 별칭으로 감싸 LiveCallBindings에 키/프로바이더/계정 다이제스트를 묶어 두며, 연결만 성공한 소켓은 계정 health에 중립으로 남깁니다.

현재 dev의 릴리스 회귀 열차·Cline 픽스처(#4390)와는 겹치지 않는 새 표면입니다. 다만 스택 계약상 #4391이 먼저 dev에 안정적으로 앉아야 하고, 그 위에 이 PR → #4395(Connections UI) 순서입니다. PR 본문도 체인을 열어 두고 부모 기준으로 리뷰하라고 적었습니다. 원격 CI는 이 시각 기준 test 2/4·macos 2/2·상위 ci가 실패한 상태이고 draft를 유지합니다. 로컬 제품 테스트는 오너 지시로 돌리지 않았고 원격 CI가 실행 게이트입니다.

스트리밍 받아쓰기는 keyed OpenAI를 400으로 거절하고 ChatGPT 구독 계정만 wss://chatgpt.com/backend-api/dictation/stream으로 보냅니다. 프레임 검증기(createDictationFrameValidator)가 pcm16·채널·VAD·세션 TTL을 빡세게 검사합니다. Live 쪽은 offer 파싱·용량(1024)·TTL 30분·사이드밴드 재연결 시 계정 불일치면 409를 돌려 소유권을 지킵니다. wp1 리뷰에서 남긴 selectOpenAiImagesProvider 재사용은 audio-upstream/audio-client에 그대로 남아 이름 혼동 이슈가 이어집니다.

src/server/audio-dictation.ts resolveDictationSocket - 업스트림 서브프로토콜에 openai-bearer.${token}을 넣는다. Bun/프록시 로그·중간 장비가 서브프로토콜을 남기면 토큰이 샐 수 있으니, 로그 redaction과 프록시 설정 주의를 문서에 한 줄 적자.
src/server/live-call-bindings.ts - 바인딩이 프로세스 메모리뿐이다. 허브 재시작·멀티 인스턴스면 별칭이 바로 404가 된다. 단일 프로세스 전제면 괜찮지만, remote hub 로드맵(#4372 계열)과 맞출지 정해야 한다.
src/server/audio-client.ts resolveAudioClient - 플랫폼 OpenAI 키 bearer와 루프백 예외를 가려내지만, 실수로 플랫폼 키를 브라우저에 넣으면 “opencodex API key required”로만 보인다. Connections 예제에 “OpenCodex 발급 키”를 더 분명히 쓰자.
src/server/ws-bridge.ts finalizeLiveSideband - liveFinish 예외를 warn만 하고 admission은 풀기에, 회계 실패가 조용히 남을 수 있다. 테스트에 finish throw 경로가 있는지는 확인하자.
원격 CI test 2/4 · macos 2/2 - draft라도 exact-head 녹색 전에는 review-ready로 올리지 말자. 실패 샤드 원인을 커밋 메시지/본문에 짧게 남기면 다음 리뷰가 빨라진다.

메인테이너의 판단이 필요한 지점

• #4391을 먼저 dev에 랜딩한 뒤 이 브랜치를 리베이스할지, draft 스택으로 통째 리뷰할지
• dictation을 ChatGPT-only로 고정할지, keyed OpenAI 스트리밍을 언제 열지
• Live 바인딩을 인메모리로 둘지, 허브 재시작을 버틸 저장이 필요한지
• bearer-in-subprotocol을 허용할지, 헤더 전용으로 다시 접을지

너의 추천
스택을 유지한 채 draft로 두고, #4391 CI·계약이 안정된 다음에 이 PR의 CI 실패부터 고치자. 서브프로토콜 토큰 노출과 인메모리 바인딩은 머지 전에 메인테이너 한 줄 확인이 필요하다. #4395 UI는 이 레이어 녹색 이후에 보자.

이 댓글은 grok-bot이 작성했습니다

[lidge-jun]

Maintainer integration record — audio stack lane, merged by @lidge-jun.

This pull request is a non-tip link in the audio stack (#4391 → #4392 → #4395). Its head carries [skip ci], so its own ci aggregate check never ran.

• CI evidence: the stack tip feat(dashboard): add dictation and live voice API controls #4395 at ec9e3734e80a32e2b60d7ace89d23de94fae0123 passed Cross-platform CI run https://github.com/lidge-jun/opencodex/actions/runs/34731037202 (conclusion success). The stack is cumulative, so this branch's content is a strict subset of what that run executed.
• Tip-only CI for this batch is an explicit repository-owner decision. MAINTAINERS.md otherwise requires a successful per-PR CI check before merge, so this is a recorded deviation rather than something the maintainer-integration clause grants on its own.
• Review state: no outstanding maintainer change requests.
• Security review: not applicable — no authentication, credential, OAuth, workflow, release-automation, or dependency-installation surface.
• Local verification on the lane: typecheck, structure:check, privacy:scan passed; focused tests passed on the files each link touches. No local full-suite run was used as evidence.

Squashing with --match-head-commit pinned to the SHA above, and with a squash message that does not carry [skip ci] so the dev-branch regression run still fires.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-13T02:07:20.831314Z	85bb3d8	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

⏳ DRAFT

• UI screenshot required.

What to do

• Add a screenshot of the UI change to the PR description.

Automatic draft conversion failed (token cannot change draft status). Please convert this pull request to a draft manually. The required enforce-target check will keep failing until every issue above is resolved.

[lidge-jun]

Maintainer integration record — audio lane root, merged by @lidge-jun.

This pull request is the dev-based root of the audio stack and now carries the whole lane: #4391 (transcription API, squashed to dev separately as afe987cff), #4392 (streaming dictation and owned live calls), and #4395 (dashboard dictation and live voice controls, squashed into this branch as 091e0c0a3). Merging this lands the remaining audio content on dev.

• Exact head at integration: 85bb3d80b426f8290424b323997480d358f87238.
• CI evidence: Cross-platform CI run https://github.com/lidge-jun/opencodex/actions/runs/34731037202 concluded success on the stack tip ec9e3734e80a32e2b60d7ace89d23de94fae0123, which is the same audio content this branch carries.
• The head above adds one merge of origin/dev on top of that verified content. dev had only advanced by the squashed feat(audio): expose standalone transcription API #4391, whose changes this branch already carried as real commits, so all 14 conflicts were the same change landing twice and were resolved to the branch side. A post-resolution check confirmed nothing dev introduced was dropped.
• Tip-only CI for this batch is an explicit repository-owner decision. MAINTAINERS.md otherwise requires a successful per-PR check before merge, so this is a recorded deviation rather than something the maintainer-integration clause grants. The dev-branch run triggered by this merge is the regression gate, and a red run is handled with a follow-up fix pull request.
• Review state: no outstanding maintainer change requests.
• Security review: not applicable — no authentication, credential, OAuth, workflow, release-automation, or dependency-installation surface.
• Local verification on the lane: typecheck, structure:check, privacy:scan passed; focused tests passed per link. No local full-suite run was used as evidence.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 85bb3d80b4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Reserve call-binding capacity before creating upstream calls

When the registry is one slot below capacity, two concurrent requests can both pass this check and then yield during credential resolution, fetch, and body reading. Both upstream calls are created, but only the first request can add its binding; the second fails at bindings.create() and returns 503 after discarding an otherwise valid SDP answer and Location, leaving an inaccessible upstream call that may consume provider resources. Reserve a binding slot atomically before the first await that creates the upstream call, then commit or release that reservation on every exit path.

Useful? React with 👍 / 👎.