AceCrush Swing-Analysis

English | 简体中文

Desktop tennis-swing auto-segmentation. Wraps a battle-tested cutting pipeline into a serviceable Python backend with a pluggable UI — the algorithm core is vendored byte-for-byte in backend/core/ (zero algorithm changes), re-copied when the underlying source changes.

┌──────────────────────────────────────────────────────────┐
│ Front-ends  (any of these, all pluggable)                │
│  • CLI        python -m backend.cli --video …            │  ← terminal UI
│  • Electron   npm run dev                                │  ← desktop UI (this repo)
│  • Browser    http://127.0.0.1:8321                      │  ← Phase C
│  • Mobile     same API + upload endpoint                 │  ← Phase C
└────────────────────────┬─────────────────────────────────┘
                         │ HTTP REST + WebSocket  (127.0.0.1:8321)
┌────────────────────────▼─────────────────────────────────┐
│ Python service  (FastAPI + uvicorn)                      │
│  service/app.py      REST routes                          │
│  service/jobs.py     JobManager + WS broadcast           │
│  service/pipeline.py shared run-pipeline                 │
│  cli.py              CLI entry — same pipeline           │
└────────────────────────┬─────────────────────────────────┘
                         │  (no algorithm changes)
┌────────────────────────▼─────────────────────────────────┐
│ core/  — three independent vendored algorithms            │
│  segment_swing.py        v2.1 wrist-signal cut pipeline  │
│  analyze_swing.py        MediaPipe 33-point once + skel  │
│  gen_skeleton_anim.py    RTMDet + RTMPose / MediaPipe    │
│                          four-quadrant skeleton animator │
└──────────────────────────────────────────────────────────┘

Design rule: the algorithm library (core/) is the single source of truth. The transport layer (REST/WS) and the interaction layer (CLI/GUI/Web) are both decoupled from it. CLI is a UI; Electron is a UI; they drive the same pipeline.

Highlights

• 🎾 Complete-cycle segmentation — every detected swing includes ready → windup → contact → follow-through phases with explicit timecodes
• 🚀 On-the-fly emit — segments appear in the UI as Pass 1 streams, not after the full video is done
• 🔌 Three pluggable UIs — terminal, Electron desktop, future browser / mobile, all sharing one REST + WebSocket contract
• 🎛 Three pluggable models — MediaPipe Pose for segmentation; RTMDet + RTMPose (or MediaPipe) for optional clip bbox + skeleton overlays
• 📦 Self-contained — three vendored algorithms + three committed models (≈160 MB) → clone-and-run, no PyPI dance for the model
• 🎬 Native video seek — the GUI plays the original video via HTTP Range, sidesteps the cv2 mp4v codec that Chromium cannot decode
• 🎞 Clip playback (plan 002) — per-clip H.264 preview (clip_NNN_h264.mp4) auto-generated by a bundled ffmpeg, so the GUI embeds each cut without re-encoding; mp4v originals stay as the canonical download artifact. Without ffmpeg the GUI falls back to "seek the original video to start_timecode" automatically.
• 📊 Dual clip progress bars (plan 003) — when clip_bbox or clip_skel is on, the GUI's progress strip grows two stacked rows: an outer clip-queue bar (已完成/已发现) and inner per-clip bars for each clip currently annotating (RTMDet / pose / both), refreshing every 5 frames. With both flags off the strip renders exactly as before — zero regression.
• ↗ F12-style detachable panels (plan 004) — the clips bar and the event log can pop out into independent OS windows (DevTools-style "Undock into separate window"), fully draggable past the main-window frame, with bidirectional state sync (clicking a clip in the detached window plays it in the main window). Panel position + size persist across restarts; existing 📌 悬浮 in-window float mode is unchanged.
• ⏳ Cancellable busy modal (plan 005) — long-running ops (export zip, clear output dir, cleanup clips, open output dir) pop a centred logo + spinner + 取消 modal that blocks the UI. The main process owns an AbortController per call; clicking 取消 aborts the underlying IO (archiver half-zip gets unlinked). Detached panel windows automatically dim + intercept clicks while the main window is busy.
• 🧩 Pure modules, composed at the pipeline layer — segmentation, detection, and pose are independent functions; you can run them together, or in any order, on the same clips

Quick Start

# CLI — fastest smoke test, no service needed
python3 -m backend.cli segment --video /abs/path/to/video.mp4 --max-frames 1500

# CLI — full pipeline with clip bbox + skeleton overlays
python3 -m backend.cli segment --video /abs/match.mp4 --save-clips --clip-bbox --clip-skel

# CLI — post-hoc annotation of already-cut clips
python3 -m backend.cli annotate --clips-dir backend/data/jobs/<id>/clips --bbox --skel

# Standalone algorithm CLIs (in backend/core/) — run a single stage without
# the service / pipeline shell:
python3 backend/core/analyze_swing.py \
    --file /abs/match.mp4 --save-clips --skel-clips --viz-full
python3 backend/core/gen_skeleton_anim.py \
    --file /abs/match.mp4 --det-model rtmdet-m-487628.onnx \
    --pose-model rtmpose-m-27c0e6.onnx

# REST service — for Electron / browser / any client
python3 -m backend.service --port 8321
# stdout last line: SWING_SERVICE_URL=http://127.0.0.1:8321

# Electron GUI — full desktop app
npm install && npm run dev

The three vendored algorithms

backend/core/ ships three independent, byte-for-byte vendored scripts. Each is runnable on its own (no service / no Electron required) and is also importable as a library:

Script	What it does	Use it when
backend/core/segment_swing.py	Pass-1 online + Pass-1.5 offline cut pipeline. Single signal: right wrist (MediaPipe idx 16). Emits SwingSegment with ready / windup / contact / follow_through phases.	You want the cut list and nothing else. This is what backend.cli segment wraps.
backend/core/analyze_swing.py	MediaPipe once → 33 points. Wrist feeds segment_swing's OnlineSegmenter; the 33 points are stored per frame so clip overlays and the full-video viz.mp4 are guaranteed 1:1 with the segments list.	You want every clip and the full video with a 33-point skeleton overlay, all in one run.
backend/core/gen_skeleton_anim.py	RTMDet (bbox) + RTMPose / MediaPipe (skeleton) four-quadrant compositor. Optional smart-zoom cropping driven by RTMDet person detection, stable ROI smoother, and auto-sizing.	You want a polished skeleton animation video independent of swing detection (no segments, just the overlay).

Documentation

The full bilingual documentation site is published on the public mirror repo at acecrush-dev.github.io/swing-analysis-app (中文) — built with VitePress, mirrored from this private repo via ./scripts/sync-to-public.sh (which writes the built dist/ into the public repo's docs/ and the README + release artifacts alongside). Markdown sources live in this repo under docs/.

Markdown sources also live in this repo under docs/ for offline reading and editing:

Chapter	Contents
00 · Introduction	What this project is, who it's for, design philosophy
01 · Getting Started	Prerequisites, install, first run (CLI / REST / GUI)
02 · Architecture	Layered design, vendor strategy, decoupling story
03 · CLI Usage	segment / annotate sub-commands, all flags, output schema, exit codes
04 · REST API	Endpoints, payloads, WebSocket event types (incl. clip.annotated), Range streaming
05 · Electron GUI	Sidecar lifecycle, dev workflow, UI layout
06 · Algorithm	v2.1 two-pass cutting pipeline + which model plays which role
07 · Troubleshooting	Common pitfalls and fixes (incl. onnxruntime, CoreML EP, RTMDet dynamic shapes)

For Chinese readers: docs/zh/ — 简体中文镜像。

Verification (fdl.mp4, first 800 frames)

[POST /api/jobs] → job_id 51b71ad9db8b
[GET  /api/jobs/51b71ad9db8b] → state=done, segments=3
[Range bytes=0-1023 /api/videos] → 206 + Content-Range: bytes 0-1023/25243119
[segments.json keys] → input / fps / total_frames / processed_frames /
                       duration_sec / wrist_detected_pct / params /
                       segments / segment_count

Clip playback (plan 002)

File	Role	Where it lives
clip_NNN.mp4	Canonical download artifact. mp4v fourcc (MPEG-4 Part 2) — Chromium cannot decode it.	backend/data/jobs/<id>/clips/
clip_NNN_h264.mp4	In-GUI embed target. H.264 + yuv420p + faststart, transcoded by bundled ffmpeg after each clip is cut.	same dir
clip_NNN_annotated.mp4	Optional bbox + skeleton overlay (when clip_bbox / clip_skel is set). Download-only, not transcoded.	same dir
clip_NNN.thumb.jpg	Lazy-generated mid-frame JPEG for the grid card.	same dir

Without ffmpeg (e.g. imageio-ffmpeg wheel failed to install and ffmpeg is not on PATH): the service leaves each clip as mp4v-only. The GUI marks such cards with ⚠ 原生格式 · 点击跳转原视频 and falls back to seeking the original video to the segment's start_timecode. The mp4v clip is still downloadable via GET /api/artifacts/{id}/clips/clip_NNN.mp4.

Endpoints (full reference in 04 · REST API):

• GET /api/jobs/{id}/clips — per-clip manifest (playable, size_bytes, thumb_ready, …)
• GET /api/jobs/{id}/clips/{seg_id}/stream — H.264 preview with HTTP Range (206)
• GET /api/jobs/{id}/clips/{seg_id}/thumbnail.jpg — lazy mid-frame JPEG
• POST /api/jobs/{id}/clips:cleanup — wipe the clips/ subdir; 409 if the job is still queued/running

License

See LICENSE.