feat: add automatic audio extraction for video downloads - #944
Open
Lcos-000 wants to merge 1 commit into
Open
Conversation
- Add ENABLE_GET_AUDIO, AUDIO_FORMAT, KEEP_ORIGINAL_VIDEO configs
- Add AbstractStoreAudio base class
- Add media_util.extract_audio_from_video using ffmpeg
- Integrate audio extraction into xhs, douyin, and bilibili crawlers
- Audio files saved to data/{platform}/audios/{id}/audio.{fmt}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
功能简介
为已支持视频下载的平台增加可选的自动音频提取能力。开启后,爬虫在下载视频的同时会自动把音频抽离成
.mp3/.aac文件,方便后续直接做 ASR 转文字。设计原则:零侵入、与主干无关、完全可选
False,不开启时现有代码路径与主干完全一致。ENABLE_GET_AUDIO单独控制,与ENABLE_GET_MEIDAS解耦。即使开启视频下载,也可以只下视频不下音频。AbstractStoreAudio与既有AbstractStoreVideo平行,保持架构一致性,不破坏现有 store 层设计。配置项
已接入平台
data/xhs/audios/{note_id}/audio.{fmt}data/douyin/audios/{aweme_id}/audio.{fmt}data/bili/audios/{aid}/audio.{fmt}使用示例
主要改动
config/base_config.py:新增 3 个音频相关配置项base/base_crawler.py:新增AbstractStoreAudio抽象类tools/media_util.py:新增异步 ffmpeg 音频提取工具,含 ffmpeg 自动查找store/{xhs,douyin,bilibili}/:各新增{platform}_store_audio.py,并注册到__init__.pymedia_platform/{xhs,douyin,bilibili}/core.py:在视频保存后,条件触发音频提取验证
uv run python -m py_compile语法检查通过uv run python main.py --help正常