Github trending [huggingface/speech-to-speech] 低延迟模块化语音流水线

[web1992]

项目地址

https://github.com/huggingface/speech-to-speech

AI 摘要

这是一个低延迟、模块化的语音智能体流水线项目，采用 VAD -> STT -> LLM -> TTS 架构，通过兼容 OpenAI Realtime 的 WebSocket API 提供服务。每个组件均可替换，LLM 支持调用云端服务或本地部署（如 vLLM、llama.cpp）。默认使用 Parakeet TDT（本地STT）、OpenAI 兼容 LLM 和 Qwen3-TTS（本地语音输出），已在数千台机器人上投入生产。一键安装即可运行，支持多语言，适用于构建全栈开源或混合部署的语音对话应用。

README 原文

 

Speech To Speech: Build voice agents with open-source models

A low-latency, fully modular voice-agent pipeline: VAD -> STT -> LLM -> TTS, exposed through an OpenAI Realtime-compatible WebSocket API. Every component is swappable. The LLM slot speaks OpenAI-compatible protocols, so you can point it at a hosted provider, at HF Inference Providers, or at a vLLM or llama.cpp server on your own hardware for a fully local, fully open stack.

This pipeline runs in production as the conversation backend for thousands of Reachy Mini robots.

Quickstart

pip install speech-to-speech
export OPENAI_API_KEY=...
speech-to-speech

This starts an OpenAI Realtime-compatible server at ws://localhost:8765/v1/realtime using Parakeet TDT for local STT, an OpenAI-compatible LLM, and Qwen3-TTS for local speech output.

From a source checkout, talk to it from a second terminal:

python scripts/listen_and_play_realtime.py --host 127.0.0.1 --port 8765

Prefer to keep the LLM on your own machine? Serve Gemma 4 wi...