Speech Fluency Pipeline — Classify → Transcribe → Regenerate

Stutter detection via spectrogram CNN, then Whisper STT and corrected TTS

A multi-stage speech pipeline that detects stuttering, transcribes the speech, and regenerates clean corrected audio — solving an audio problem by turning it into an image problem.

---

Pipeline

flowchart LR
    A["Audio file"] --> SP["Convert to spectrogram image<br/>(librosa)"]
    SP --> CNN["ResNet18 classifier<br/>fluency: normal / stuttered"]
    A --> STT["Whisper STT<br/>precise transcription"]
    STT --> TTS["pyttsx3 TTS<br/>regenerate clean audio"]
    CNN --> OUT["Fluency label"]
    TTS --> OUT2["Corrected audio file"]

Loading

The core idea: audio classification is reframed as image classification — each clip becomes a spectrogram that a ResNet18 CNN classifies for fluency. Whisper then transcribes the original speech, and pyttsx3 regenerates a clean version from the (corrected) text.

Setup

git clone https://github.com/YazanAi-Dev3/Audio-Classification-ResNet.git
cd Audio-Classification-ResNet

# FFmpeg is required (Windows): winget install Gyan.FFmpeg
python -m venv venv   # activate, then:
pip install -r requirements.txt

Run the full workflow in Audio_Classification_Demo.ipynb.

Tech Stack

Python · PyTorch / torchvision (ResNet18) · OpenAI Whisper (STT) · pyttsx3 (TTS) · librosa (spectrograms)

License

MIT — see LICENSE.