更新
This commit is contained in:
@@ -42,17 +42,28 @@
|
||||
|
||||
| 模块 | 技术选择 | 备选方案 |
|
||||
|------|----------|----------|
|
||||
| **前端框架** | Next.js 14 | Vue 3 + Vite |
|
||||
| **UI 组件库** | Tailwind + shadcn/ui | Ant Design |
|
||||
| **后端框架** | FastAPI | Flask |
|
||||
| **任务队列** | Celery + Redis | RQ / Dramatiq |
|
||||
| **唇形同步** | **LatentSync 1.6** | MuseTalk / Wav2Lip |
|
||||
| **TTS 配音** | EdgeTTS | CosyVoice |
|
||||
| **声音克隆** | **Qwen3-TTS 1.7B** ✅ | GPT-SoVITS |
|
||||
| **视频处理** | FFmpeg | MoviePy |
|
||||
| **自动发布** | social-auto-upload | 自行实现 |
|
||||
| **数据库** | SQLite → PostgreSQL | MySQL |
|
||||
| **文件存储** | 本地 / MinIO | 阿里云 OSS |
|
||||
| **前端框架** | Next.js 16 | Vue 3 + Vite |
|
||||
| **UI 组件库** | TailwindCSS (自定义组件) | Ant Design |
|
||||
| **后端框架** | FastAPI | Flask |
|
||||
| **任务队列** | FastAPI BackgroundTasks (asyncio) | Celery + Redis |
|
||||
| **唇形同步** | **LatentSync 1.6** | MuseTalk / Wav2Lip |
|
||||
| **TTS 配音** | EdgeTTS | CosyVoice |
|
||||
| **声音克隆** | **Qwen3-TTS 1.7B** ✅ | GPT-SoVITS |
|
||||
| **视频处理** | FFmpeg | MoviePy |
|
||||
| **自动发布** | Playwright | 自行实现 |
|
||||
| **数据库** | Supabase (PostgreSQL) | MySQL |
|
||||
| **文件存储** | Supabase Storage | 阿里云 OSS |
|
||||
|
||||
> **修正 (18:10)**:当前实现采用 Next.js 16、FastAPI BackgroundTasks 与 Supabase Storage/Auth,自动发布基于 Playwright。
|
||||
|
||||
---
|
||||
|
||||
## ✅ 现状补充 (Day 17)
|
||||
|
||||
- 前端已拆分为组件化结构(`components/home/`),主页面逻辑集中。
|
||||
- 通用工具 `media.ts` 统一处理 API Base / 资源 URL / 日期格式化。
|
||||
- 作品预览弹窗统一样式,并支持素材/发布预览复用。
|
||||
- 标题/字幕预览按素材分辨率缩放,效果更接近成片。
|
||||
|
||||
---
|
||||
|
||||
@@ -60,24 +71,11 @@
|
||||
|
||||
### 阶段一:核心功能验证 (MVP)
|
||||
|
||||
> **目标**:验证 MuseTalk + EdgeTTS 效果,跑通端到端流程
|
||||
> **目标**:验证 LatentSync + EdgeTTS 效果,跑通端到端流程
|
||||
|
||||
#### 1.1 环境搭建
|
||||
|
||||
```bash
|
||||
# 创建项目目录
|
||||
mkdir TalkingHeadAgent
|
||||
cd TalkingHeadAgent
|
||||
|
||||
# 克隆 MuseTalk
|
||||
git clone https://github.com/TMElyralab/MuseTalk.git
|
||||
|
||||
# 安装依赖
|
||||
cd MuseTalk
|
||||
pip install -r requirements.txt
|
||||
|
||||
# 下载模型权重 (按官方文档)
|
||||
```
|
||||
#### 1.1 环境搭建
|
||||
|
||||
参考 `models/LatentSync/DEPLOY.md` 完成 LatentSync 环境与权重部署。
|
||||
|
||||
#### 1.2 集成 EdgeTTS
|
||||
|
||||
@@ -98,13 +96,13 @@ async def text_to_speech(text: str, voice: str = "zh-CN-YunxiNeural", output_pat
|
||||
# test_pipeline.py
|
||||
"""
|
||||
1. 文案 → EdgeTTS → 音频
|
||||
2. 静态视频 + 音频 → MuseTalk → 口播视频
|
||||
2. 静态视频 + 音频 → LatentSync → 口播视频
|
||||
3. 添加字幕 → FFmpeg → 最终视频
|
||||
"""
|
||||
```
|
||||
|
||||
#### 1.4 验证标准
|
||||
- [ ] MuseTalk 能正常推理
|
||||
- [ ] LatentSync 能正常推理
|
||||
- [ ] 唇形与音频同步率 > 90%
|
||||
- [ ] 单个视频生成时间 < 2 分钟
|
||||
|
||||
@@ -145,22 +143,16 @@ backend/
|
||||
| `/api/materials` | POST | 上传素材视频 | ✅ |
|
||||
| `/api/materials` | GET | 获取素材列表 | ✅ |
|
||||
| `/api/videos/generate` | POST | 创建视频生成任务 | ✅ |
|
||||
| `/api/tasks/{id}` | GET | 查询任务状态 | ✅ |
|
||||
| `/api/videos/{id}/download` | GET | 下载生成的视频 | ✅ |
|
||||
| `/api/videos/tasks/{id}` | GET | 查询任务状态 | ✅ |
|
||||
| `/api/videos/generated` | GET | 获取历史作品列表 | ✅ |
|
||||
| `/api/publish` | POST | 发布到社交平台 | ✅ |
|
||||
|
||||
#### 2.3 Celery 任务定义
|
||||
|
||||
```python
|
||||
# tasks/celery_tasks.py
|
||||
@celery.task
|
||||
def generate_video_task(material_id: str, text: str, voice: str):
|
||||
# 1. TTS 生成音频
|
||||
# 2. MuseTalk 唇形同步
|
||||
# 3. FFmpeg 添加字幕
|
||||
# 4. 保存并返回视频 URL
|
||||
pass
|
||||
```
|
||||
#### 2.3 BackgroundTasks 任务定义
|
||||
|
||||
```python
|
||||
# app/api/videos.py
|
||||
background_tasks.add_task(_process_video_generation, task_id, req, user_id)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -183,9 +175,9 @@ def generate_video_task(material_id: str, text: str, voice: str):
|
||||
# 创建 Next.js 项目
|
||||
npx create-next-app@latest frontend --typescript --tailwind --app
|
||||
|
||||
# 安装依赖
|
||||
cd frontend
|
||||
npm install @tanstack/react-query axios
|
||||
# 安装依赖
|
||||
cd frontend
|
||||
npm install axios swr
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user