更新
This commit is contained in:
Kevin Wong
@@ -117,6 +117,9 @@ backend/
|
||||
- `WEIXIN_USER_AGENT` / `WEIXIN_LOCALE` / `WEIXIN_TIMEZONE_ID`
|
||||
- `WEIXIN_FORCE_SWIFTSHADER`
|
||||
- `WEIXIN_TRANSCODE_MODE` (reencode/faststart/off)
|
||||
- `CORS_ORIGINS` (CORS 白名单，默认 *)
|
||||
- `SUPABASE_STORAGE_LOCAL_PATH` (本地存储路径)
|
||||
- `DOUYIN_COOKIE` (抖音视频下载 Cookie)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -189,6 +189,9 @@ cp .env.example .env
|
||||
| `WEIXIN_TIMEZONE_ID` | Asia/Shanghai | 视频号时区 |
|
||||
| `WEIXIN_FORCE_SWIFTSHADER` | true | 强制软件 WebGL，避免 context lost |
|
||||
| `WEIXIN_TRANSCODE_MODE` | reencode | 上传前转码 (reencode/faststart/off) |
|
||||
| `CORS_ORIGINS` | `*` | CORS 允许源 (生产环境建议白名单) |
|
||||
| `SUPABASE_STORAGE_LOCAL_PATH` | 默认路径 | Supabase 本地存储路径 |
|
||||
| `DOUYIN_COOKIE` | 空 | 抖音视频下载 Cookie (文案提取功能) |
|
||||
|
||||
---
|
||||
|
||||
|
||||
66
Docs/DevLogs/Day20.md
Normal file
66
Docs/DevLogs/Day20.md
Normal file
@@ -0,0 +1,66 @@
|
||||
## 🔧 代码质量与安全优化 (13:30)
|
||||
|
||||
### 概述
|
||||
本日进行项目全面代码审查与优化，共处理 27 项优化点，完成 18 项核心修复。
|
||||
|
||||
### 已完成优化
|
||||
|
||||
#### 功能性修复
|
||||
- [x] **P0-1**: LatentSync 回退逻辑空实现 → 改为 `raise RuntimeError`
|
||||
- [x] **P1-1**: 任务状态接口无用户归属校验 → 添加用户认证依赖
|
||||
- [x] **P1-2**: 前端 User 类型定义重复 → 统一到 `shared/types/user.ts`
|
||||
|
||||
#### 性能优化
|
||||
- [x] **P1-3**: 参考音频列表 N+1 查询 → 使用 `asyncio.gather` 并发
|
||||
- [x] **P1-4**: 视频上传整读内存 → 新增 `upload_file_from_path` 流式处理
|
||||
- [x] **P1-5**: async 路由内同步阻塞 → `httpx.AsyncClient` 替换 `requests`
|
||||
- [x] **P2-2**: GLM 服务同步调用 → `asyncio.to_thread` 包装
|
||||
- [x] **P2-3**: Remotion 渲染启动慢 → 预编译 JS + `build:render` 脚本
|
||||
|
||||
#### 安全修复
|
||||
- [x] **P1-8**: 硬编码 Cookie → 移至环境变量 `DOUYIN_COOKIE`
|
||||
- [x] **P1-9**: 请求日志打印完整 headers → 敏感信息脱敏
|
||||
- [x] **P2-10**: ffprobe 使用 `shell=True` → 改为参数列表
|
||||
- [x] **P2-11**: CORS 配置 `*` + credentials → 从 `CORS_ORIGINS` 环境变量读取
|
||||
|
||||
#### 配置优化
|
||||
- [x] **P2-5**: 存储服务硬编码路径 → 环境变量 `SUPABASE_STORAGE_LOCAL_PATH`
|
||||
- [x] **P3-3**: Remotion `execSync` 同步调用 → promisified `exec` 异步
|
||||
- [x] **P3-5**: LatentSync 相对路径 → 基于 `__file__` 绝对路径
|
||||
|
||||
### 暂不处理（收益有限）
|
||||
- [~] **P1-6**: useHomeController 超大文件 (884行)
|
||||
- [~] **P1-7**: 抖音/微信上传器重复代码（流程差异大）
|
||||
|
||||
### 低优先级（后续处理）
|
||||
- [~] **P2-6~P2-9**: API 转发壳、前端 API 客户端混用、ESLint、重复逻辑
|
||||
- [~] **P3-1~P3-4**: 阻塞式交互、Modal 过大、样式兼容层
|
||||
|
||||
### 涉及文件
|
||||
- `backend/app/services/latentsync_service.py` - 回退逻辑
|
||||
- `backend/app/modules/videos/router.py` - 任务状态认证
|
||||
- `backend/app/modules/tools/router.py` - httpx 异步、Cookie 配置化
|
||||
- `backend/app/services/glm_service.py` - 异步包装
|
||||
- `backend/app/services/storage.py` - 流式上传、路径配置化
|
||||
- `backend/app/services/video_service.py` - ffprobe 安全调用
|
||||
- `backend/app/main.py` - CORS 配置、日志脱敏
|
||||
- `backend/app/core/config.py` - 新增配置项
|
||||
- `remotion/render.ts` - 异步 exec
|
||||
- `remotion/package.json` - build:render 脚本
|
||||
- `models/LatentSync/scripts/server.py` - 绝对路径
|
||||
- `frontend/src/shared/types/user.ts` - 统一类型定义
|
||||
|
||||
### 新增环境变量
|
||||
```bash
|
||||
# .env 新增配置（均有默认值，无需必填）
|
||||
CORS_ORIGINS=* # CORS 白名单
|
||||
SUPABASE_STORAGE_LOCAL_PATH=/path/to/... # 本地存储路径
|
||||
DOUYIN_COOKIE=... # 抖音视频下载 Cookie
|
||||
```
|
||||
|
||||
### 重启要求
|
||||
```bash
|
||||
pm2 restart vigent2-backend
|
||||
pm2 restart vigent2-latentsync
|
||||
# Remotion 已自动编译
|
||||
```
|
||||
@@ -26,8 +26,11 @@
|
||||
| 🔥 **High** | `Docs/task_complete.md` | **(任务总览)** 更新 `[x]`、进度条、时间线 |
|
||||
| ⚡ **Med** | `README.md` | **(项目主页)** 功能特性、技术栈、最新截图 |
|
||||
| ⚡ **Med** | `Docs/DEPLOY_MANUAL.md` | **(部署手册)** 环境变量、依赖包、启动命令变更 |
|
||||
| ⚡ **Med** | `Docs/BACKEND_DEV.md` | **(后端规范)** 接口契约、模块划分、环境变量 |
|
||||
| ⚡ **Med** | `Docs/BACKEND_README.md` | **(后端文档)** 接口说明、架构设计 |
|
||||
| ⚡ **Med** | `Docs/FRONTEND_DEV.md` | **(前端规范)** API封装、日期格式化、新页面规范 |
|
||||
| ⚡ **Med** | `Docs/FRONTEND_README.md` | **(前端文档)** 功能说明、页面变更 |
|
||||
| 🧊 **Low** | `Docs/*_DEPLOY.md` | **(子系统部署)** LatentSync/Qwen3/字幕等独立部署文档 |
|
||||
| 🧊 **Low** | `Docs/implementation_plan.md` | **(实施计划)** 核对计划与实际实现的差异 |
|
||||
| 🧊 **Low** | `Docs/architecture_plan.md` | **(前端架构)** 拆分计划与阶段目标 |
|
||||
|
||||
@@ -212,11 +215,17 @@
|
||||
ViGent2/Docs/
|
||||
├── task_complete.md # 任务总览（仅按需更新）
|
||||
├── Doc_Rules.md # 本文件
|
||||
├── BACKEND_DEV.md # 后端开发规范
|
||||
├── BACKEND_README.md # 后端功能文档
|
||||
├── FRONTEND_DEV.md # 前端开发规范
|
||||
├── FRONTEND_README.md # 前端功能文档
|
||||
├── architecture_plan.md # 前端拆分计划
|
||||
├── implementation_plan.md # 实施计划
|
||||
├── DEPLOY_MANUAL.md # 部署手册
|
||||
├── SUPABASE_DEPLOY.md # Supabase 部署文档
|
||||
├── LatentSync_DEPLOY.md # LatentSync 部署文档
|
||||
├── QWEN3_TTS_DEPLOY.md # 声音克隆部署文档
|
||||
├── SUBTITLE_DEPLOY.md # 字幕系统部署文档
|
||||
└── DevLogs/
|
||||
├── Day1.md # 开发日志
|
||||
└── ...
|
||||
@@ -316,4 +325,4 @@ ViGent2/Docs/
|
||||
|
||||
---
|
||||
|
||||
**最后更新**：2026-02-04
|
||||
**最后更新**：2026-02-07
|
||||
|
||||
@@ -233,6 +233,12 @@ import { formatDate } from '@/shared/lib/media';
|
||||
- `features/*/ui`：功能 UI 组件
|
||||
- `shared/`：通用工具、通用 hooks、API 实例
|
||||
|
||||
## 类型定义规范
|
||||
|
||||
- 通用实体类型（如 User, Account, Video）统一放置在 `src/shared/types/`。
|
||||
- 特定业务类型放在 feature 目录下的 types.ts 或 model 中。
|
||||
- **禁止**在多个地方重复定义 User 接口，统一引用 `import { User } from '@/shared/types/user';`。
|
||||
|
||||
---
|
||||
|
||||
## 用户偏好持久化
|
||||
|
||||
@@ -52,6 +52,9 @@ cd /home/rongye/ProgramFiles/ViGent2/remotion
|
||||
|
||||
# 安装依赖
|
||||
npm install
|
||||
|
||||
# 预编译渲染脚本 (生产环境必须)
|
||||
npm run build:render
|
||||
```
|
||||
|
||||
### 步骤 3: 重启后端服务
|
||||
|
||||
@@ -369,6 +369,17 @@ cp -r SuperIPAgent/social-auto-upload backend/social_upload
|
||||
|
||||
---
|
||||
|
||||
### 阶段二十：代码质量与安全优化 (Day 20) ✅
|
||||
|
||||
> **目标**：全面提升代码健壮性、安全性与配置灵活性
|
||||
|
||||
- [x] **安全性修复**：硬编码 Cookie/Key 移除，ffprobe 安全调用，日志脱敏
|
||||
- [x] **配置化改造**：存储路径、CORS、录屏开关全面环境变量化
|
||||
- [x] **性能优化**：API 异步改造 (httpx/asyncio)，大文件流式上传
|
||||
- [x] **构建优化**：Remotion 预编译，统一启动脚本 `run_backend.sh`
|
||||
|
||||
---
|
||||
|
||||
## 验证计划
|
||||
|
||||
### 阶段一验证
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# ViGent2 开发任务清单 (Task Log)
|
||||
|
||||
**项目**: ViGent2 数字人口播视频生成系统
|
||||
**进度**: 100% (Day 19 - 自动发布稳定性与发布体验优化)
|
||||
**更新时间**: 2026-02-06
|
||||
**进度**: 100% (Day 20 - 代码质量与安全优化)
|
||||
**更新时间**: 2026-02-07
|
||||
|
||||
---
|
||||
|
||||
@@ -10,7 +10,14 @@
|
||||
|
||||
> 这里记录了每一天的核心开发内容与 milestone。
|
||||
|
||||
### Day 19: 自动发布稳定性与发布体验优化 (Current) 🚀
|
||||
### Day 20: 代码质量与安全优化 (Current)
|
||||
- [x] **功能性修复**: LatentSync 回退逻辑、任务状态接口认证、User 类型统一。
|
||||
- [x] **性能优化**: N+1 查询修复、视频上传流式处理、httpx 异步替换、GLM 异步包装。
|
||||
- [x] **安全修复**: 硬编码 Cookie 配置化、日志敏感信息脱敏、ffprobe 安全调用、CORS 配置化。
|
||||
- [x] **配置优化**: 存储路径环境变量化、Remotion 预编译加速、LatentSync 绝对路径。
|
||||
- [x] **文档更新**: 更新 Doc_Rules.md 清单，补齐后端与部署文档；更新 SUBTITLE_DEPLOY.md, FRONTEND_DEV.md, implementation_plan.md。
|
||||
|
||||
### Day 19: 自动发布稳定性与发布体验优化 🚀
|
||||
- [x] **抖音发布稳定性**: 上传入口、封面流程、发布重试、登录失效识别与网络失败快速返回全面增强。
|
||||
- [x] **视频号发布修复**: 标题+标签统一写入“视频描述”，`post_create` 成功信号快速判定，超时改为失败返回。
|
||||
- [x] **成功截图闭环**: 抖音/视频号发布成功截图接入前端，支持用户隔离存储与鉴权访问。
|
||||
|
||||
@@ -66,3 +66,7 @@ ADMIN_PASSWORD=lam1988324
|
||||
# 智谱 GLM API 配置 (用于生成标题和标签)
|
||||
GLM_API_KEY=32440cd3f3444d1f8fe721304acea8bd.YXNLrk7eIJMKcg4t
|
||||
GLM_MODEL=glm-4.7-flash
|
||||
|
||||
# =============== 抖音视频下载 Cookie ===============
|
||||
# 用于从抖音 URL 提取视频文案功能，会过期需要定期更新
|
||||
DOUYIN_COOKIE=douyin.com; device_web_cpu_core=10; device_web_memory_size=8; __ac_nonce=06760391f00b9b51264ae; __ac_signature=_02B4Z6wo00f019a5ceAAAIDAhEZR-X3jjWfWmXVAAJLXd4; ttwid=1%7C7MTKBSMsP4eOv9h5NAh8p0E-NYIud09ftNmB0mjLpWc%7C1734359327%7C8794abeabbd47447e1f56e5abc726be089f2a0344d6343b5f75f23e7b0f0028f; UIFID_TEMP=0de8750d2b188f4235dbfd208e44abbb976428f0720eb983255afefa45d39c0c6532e1d4768dd8587bf919f866ff1396912bcb2af71efee56a14a2a9f37b74010d0a0413795262f6d4afe02a032ac7ab; s_v_web_id=verify_m4r4ribr_c7krmY1z_WoeI_43po_ATpO_I4o8U1bex2D7; hevc_supported=true; home_can_add_dy_2_desktop=%220%22; dy_swidth=2560; dy_sheight=1440; stream_recommend_feed_params=%22%7B%5C%22cookie_enabled%5C%22%3Atrue%2C%5C%22screen_width%5C%22%3A2560%2C%5C%22screen_height%5C%22%3A1440%2C%5C%22browser_online%5C%22%3Atrue%2C%5C%22cpu_core_num%5C%22%3A10%2C%5C%22device_memory%5C%22%3A8%2C%5C%22downlink%5C%22%3A10%2C%5C%22effective_type%5C%22%3A%5C%224g%5C%22%2C%5C%22round_trip_time%5C%22%3A50%7D%22; strategyABtestKey=%221734359328.577%22; csrf_session_id=2f53aed9aa6974e83aa9a1014180c3a4; fpk1=U2FsdGVkX1/IpBh0qdmlKAVhGyYHgur4/VtL9AReZoeSxadXn4juKvsakahRGqjxOPytHWspYoBogyhS/V6QSw==; fpk2=0845b309c7b9b957afd9ecf775a4c21f; passport_csrf_token=d80e0c5b2fa2328219856be5ba7e671e; passport_csrf_token_default=d80e0c5b2fa2328219856be5ba7e671e; odin_tt=3c891091d2eb0f4718c1d5645bc4a0017032d4d5aa989decb729e9da2ad570918cbe5e9133dc6b145fa8c758de98efe32ff1f81aa0d611e838cc73ab08ef7d3f6adf66ab4d10e8372ddd628f94f16b8e; volume_info=%7B%22isUserMute%22%3Afalse%2C%22isMute%22%3Afalse%2C%22volume%22%3A0.5%7D; bd_ticket_guard_client_web_domain=2; FORCE_LOGIN=%7B%22videoConsumedRemainSeconds%22%3A180%7D; UIFID=0de8750d2b188f4235dbfd208e44abbb976428f0720eb983255afefa45d39c0c6532e1d4768dd8587bf919f866ff139655a3c2b735923234f371c699560c657923fd3d6c5b63ab7bb9b83423b6cb4787e2ce66a7fbc4ecb24c8570f520fe6de068bbb95115023c0c6c1b6ee31b49fb7e3996fb8349f43a3fd8b7a61cd9e18e8fe65eb6a7c13de4c0960d84e344b644725db3eb2fa6b7caf821de1b50527979f2; is_dash_user=1; biz_trace_id=b57a241f; bd_ticket_guard_client_data=eyJiZC10aWNrZXQtZ3VhcmQtdmVyc2lvbiI6MiwiYmQtdGlja2V0LWd1YXJkLWl0ZXJhdGlvbi12ZXJzaW9uIjoxLCJiZC10aWNrZXQtZ3VhcmQtcmVlLXB1YmxpYy1rZXkiOiJCTEo2R0lDalVoWW1XcHpGOFdrN0Vrc0dXcCtaUzNKY1g4NGNGY2k0TTl1TEowNjdUb21mbFU5aDdvWVBGamhNRWNRQWtKdnN1MnM3RmpTWnlJQXpHMjA9IiwiYmQtdGlja2V0LWd1YXJkLXdlYi12ZXJzaW9uIjoyfQ%3D%3D; download_guide=%221%2F20241216%2F0%22; sdk_source_info=7e276470716a68645a606960273f276364697660272927676c715a6d6069756077273f276364697660272927666d776a68605a607d71606b766c6a6b5a7666776c7571273f275e58272927666a6b766a69605a696c6061273f27636469766027292762696a6764695a7364776c6467696076273f275e5827292771273f273d33323131333c3036313632342778; bit_env=RiOY4jzzpxZoVCl6zdVSVhVRjdwHRTxqcqWdqMBZLPGjMdB4Tax1kAELHNTVAAh72KuhumewE4Lq6f0-VJ2UpJrkrhSxoPw9LUb3zQrq1OSwbeSPHkRlRgRQvO89sItdGUyq1oFr0XyRCnMYG87KSeWyc4x0czGR0o50hTDoDLG5rJVoRcdQOLvjiAegsqyytKF59sPX_QM9qffK2SqYsg0hCggURc_AI6kguDDE5DvG0bnyz1utw4z1eEnIoLrkGDqzqBZj4dOAr0BVU6ofbsS-pOQ2u2PM1dLP9FlBVBlVaqYVgHJeSLsR5k76BRTddUjTb4zEilVIEwAMJWGN4I1BxVt6fC9B5tBQpuT0lj3n3eKXCKXZsd8FrEs5_pbfDsxV-e_WMiXI2ff4qxiTC0U73sfo9OpicKICtZjdq8qsHxJuu6wVR36zvXeL2Wch5C6MzprNvkivv0l8nbh2mSgy1nabZr3dmU6NcR-Bg3Q3xTWUlR9aAUmpopC-cNuXjgLpT-Lw1AYGilSUnCvosth1Gfypq-b0MpgmdSDgTrQ%3D; gulu_source_res=eyJwX2luIjoiMDhjOGQ3ZTJiODQyNjZkZWI5Y2VkMGJiODNlNmY1ZWY0ZjMyNTE2ZmYyZjAzNDMzZjI0OWU1Y2Q1NTczNTk5NyJ9; passport_auth_mix_state=hp9bc3dgb1tm5wd8p82zawus27g0e3ue; IsDouyinActive=false
|
||||
|
||||
@@ -76,6 +76,12 @@ class Settings(BaseSettings):
|
||||
GLM_API_KEY: str = ""
|
||||
GLM_MODEL: str = "glm-4.7-flash"
|
||||
|
||||
# CORS 配置 (逗号分隔的域名列表，* 表示允许所有)
|
||||
CORS_ORIGINS: str = "*"
|
||||
|
||||
# 抖音 Cookie (用于视频下载功能，会过期需要定期更新)
|
||||
DOUYIN_COOKIE: str = ""
|
||||
|
||||
@property
|
||||
def LATENTSYNC_DIR(self) -> Path:
|
||||
"""LatentSync 目录路径 (动态计算)"""
|
||||
|
||||
@@ -19,10 +19,27 @@ import time
|
||||
import traceback
|
||||
|
||||
class LoggingMiddleware(BaseHTTPMiddleware):
|
||||
# 敏感 header 名称列表（小写）
|
||||
SENSITIVE_HEADERS = {'authorization', 'cookie', 'set-cookie', 'x-api-key', 'api-key'}
|
||||
|
||||
def _sanitize_headers(self, headers: dict) -> dict:
|
||||
"""脱敏处理请求头，隐藏敏感信息"""
|
||||
sanitized = {}
|
||||
for key, value in headers.items():
|
||||
if key.lower() in self.SENSITIVE_HEADERS:
|
||||
# 显示前8个字符 + 掩码
|
||||
if len(value) > 8:
|
||||
sanitized[key] = value[:8] + "..." + f"[{len(value)} chars]"
|
||||
else:
|
||||
sanitized[key] = "[REDACTED]"
|
||||
else:
|
||||
sanitized[key] = value
|
||||
return sanitized
|
||||
|
||||
async def dispatch(self, request: Request, call_next):
|
||||
start_time = time.time()
|
||||
logger.info(f"START Request: {request.method} {request.url}")
|
||||
logger.info(f"HEADERS: {dict(request.headers)}")
|
||||
logger.debug(f"HEADERS: {self._sanitize_headers(dict(request.headers))}")
|
||||
try:
|
||||
response = await call_next(request)
|
||||
process_time = time.time() - start_time
|
||||
@@ -63,10 +80,15 @@ async def unhandled_exception_handler(request: Request, exc: Exception):
|
||||
content=error_response("服务器内部错误", 500),
|
||||
)
|
||||
|
||||
# CORS 配置：从环境变量读取允许的域名
|
||||
# 当使用 credentials 时，不能使用 * 通配符
|
||||
cors_origins = settings.CORS_ORIGINS.split(",") if settings.CORS_ORIGINS != "*" else ["*"]
|
||||
allow_credentials = settings.CORS_ORIGINS != "*" # 使用 * 时不能 allow_credentials
|
||||
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"],
|
||||
allow_credentials=True,
|
||||
allow_origins=cors_origins,
|
||||
allow_credentials=allow_credentials,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
@@ -249,16 +249,17 @@ async def list_ref_audios(user: dict = Depends(get_current_user)):
|
||||
# 列出用户目录下的文件
|
||||
files = await storage_service.list_files(BUCKET_REF_AUDIOS, user_id)
|
||||
|
||||
# 过滤出 .wav 文件并获取对应的 metadata
|
||||
items = []
|
||||
for f in files:
|
||||
# 过滤出 .wav 文件
|
||||
wav_files = [f for f in files if f.get("name", "").endswith(".wav")]
|
||||
|
||||
if not wav_files:
|
||||
return success_response(RefAudioListResponse(items=[]).model_dump())
|
||||
|
||||
# 并发获取所有 metadata 和签名 URL
|
||||
async def fetch_audio_info(f):
|
||||
"""获取单个音频的信息（metadata + signed URL）"""
|
||||
name = f.get("name", "")
|
||||
if not name.endswith(".wav"):
|
||||
continue
|
||||
|
||||
storage_path = f"{user_id}/{name}"
|
||||
|
||||
# 尝试读取 metadata
|
||||
metadata_name = name.replace(".wav", ".json")
|
||||
metadata_path = f"{user_id}/{metadata_name}"
|
||||
|
||||
@@ -271,7 +272,7 @@ async def list_ref_audios(user: dict = Depends(get_current_user)):
|
||||
# 获取 metadata 内容
|
||||
metadata_url = await storage_service.get_signed_url(BUCKET_REF_AUDIOS, metadata_path)
|
||||
import httpx
|
||||
async with httpx.AsyncClient() as client:
|
||||
async with httpx.AsyncClient(timeout=5.0) as client:
|
||||
resp = await client.get(metadata_url)
|
||||
if resp.status_code == 200:
|
||||
metadata = resp.json()
|
||||
@@ -280,7 +281,7 @@ async def list_ref_audios(user: dict = Depends(get_current_user)):
|
||||
created_at = metadata.get("created_at", 0)
|
||||
original_filename = metadata.get("original_filename", "")
|
||||
except Exception as e:
|
||||
logger.warning(f"读取 metadata 失败: {e}")
|
||||
logger.debug(f"读取 metadata 失败: {e}")
|
||||
# 从文件名提取时间戳
|
||||
try:
|
||||
created_at = int(name.split("_")[0])
|
||||
@@ -299,17 +300,21 @@ async def list_ref_audios(user: dict = Depends(get_current_user)):
|
||||
if match:
|
||||
display_name = match.group(1)
|
||||
|
||||
items.append(RefAudioResponse(
|
||||
return RefAudioResponse(
|
||||
id=storage_path,
|
||||
name=display_name,
|
||||
path=signed_url,
|
||||
ref_text=ref_text,
|
||||
duration_sec=duration_sec,
|
||||
created_at=created_at
|
||||
))
|
||||
)
|
||||
|
||||
# 使用 asyncio.gather 并发获取所有音频信息
|
||||
import asyncio
|
||||
items = await asyncio.gather(*[fetch_audio_info(f) for f in wav_files])
|
||||
|
||||
# 按创建时间倒序排列
|
||||
items.sort(key=lambda x: x.created_at, reverse=True)
|
||||
items = sorted(items, key=lambda x: x.created_at, reverse=True)
|
||||
|
||||
return success_response(RefAudioListResponse(items=items).model_dump())
|
||||
|
||||
|
||||
@@ -210,6 +210,8 @@ async def download_douyin_manual(url: str, temp_dir: Path, timestamp: int) -> Op
|
||||
手动下载抖音视频 (Fallback logic - Ported from SuperIPAgent/douyinDownloader)
|
||||
使用特定的 User Profile URL 和硬编码 Cookie 绕过反爬
|
||||
"""
|
||||
import httpx
|
||||
|
||||
logger.info(f"[SuperIPAgent] Starting download for: {url}")
|
||||
|
||||
try:
|
||||
@@ -218,9 +220,11 @@ async def download_douyin_manual(url: str, temp_dir: Path, timestamp: int) -> Op
|
||||
"user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
|
||||
}
|
||||
|
||||
# 如果是短链或重定向
|
||||
resp = requests.get(url, headers=headers, allow_redirects=True, timeout=10)
|
||||
final_url = resp.url
|
||||
# 如果是短链或重定向 - 使用异步 httpx
|
||||
async with httpx.AsyncClient(follow_redirects=True, timeout=10.0) as client:
|
||||
resp = await client.get(url, headers=headers)
|
||||
final_url = str(resp.url)
|
||||
|
||||
logger.info(f"[SuperIPAgent] Final URL: {final_url}")
|
||||
|
||||
modal_id = None
|
||||
@@ -238,16 +242,21 @@ async def download_douyin_manual(url: str, temp_dir: Path, timestamp: int) -> Op
|
||||
# 使用特定用户的 Profile 页 + modal_id 参数，配合特定 Cookie
|
||||
target_url = f"https://www.douyin.com/user/MS4wLjABAAAAN_s_hups7LD0N4qnrM3o2gI0vuG3pozNaEolz2_py3cHTTrpVr1Z4dukFD9SOlwY?from_tab_name=main&modal_id={modal_id}"
|
||||
|
||||
# 3. 使用硬编码 Cookie (Copy from SuperIPAgent)
|
||||
# 3. 使用配置的 Cookie (从环境变量 DOUYIN_COOKIE 读取)
|
||||
from app.core.config import settings
|
||||
if not settings.DOUYIN_COOKIE:
|
||||
logger.warning("[SuperIPAgent] DOUYIN_COOKIE 未配置，视频下载可能失败")
|
||||
|
||||
headers_with_cookie = {
|
||||
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
|
||||
"cookie": "douyin.com; device_web_cpu_core=10; device_web_memory_size=8; __ac_nonce=06760391f00b9b51264ae; __ac_signature=_02B4Z6wo00f019a5ceAAAIDAhEZR-X3jjWfWmXVAAJLXd4; ttwid=1%7C7MTKBSMsP4eOv9h5NAh8p0E-NYIud09ftNmB0mjLpWc%7C1734359327%7C8794abeabbd47447e1f56e5abc726be089f2a0344d6343b5f75f23e7b0f0028f; UIFID_TEMP=0de8750d2b188f4235dbfd208e44abbb976428f0720eb983255afefa45d39c0c6532e1d4768dd8587bf919f866ff1396912bcb2af71efee56a14a2a9f37b74010d0a0413795262f6d4afe02a032ac7ab; s_v_web_id=verify_m4r4ribr_c7krmY1z_WoeI_43po_ATpO_I4o8U1bex2D7; hevc_supported=true; home_can_add_dy_2_desktop=%220%22; dy_swidth=2560; dy_sheight=1440; stream_recommend_feed_params=%22%7B%5C%22cookie_enabled%5C%22%3Atrue%2C%5C%22screen_width%5C%22%3A2560%2C%5C%22screen_height%5C%22%3A1440%2C%5C%22browser_online%5C%22%3Atrue%2C%5C%22cpu_core_num%5C%22%3A10%2C%5C%22device_memory%5C%22%3A8%2C%5C%22downlink%5C%22%3A10%2C%5C%22effective_type%5C%22%3A%5C%224g%5C%22%2C%5C%22round_trip_time%5C%22%3A50%7D%22; strategyABtestKey=%221734359328.577%22; csrf_session_id=2f53aed9aa6974e83aa9a1014180c3a4; fpk1=U2FsdGVkX1/IpBh0qdmlKAVhGyYHgur4/VtL9AReZoeSxadXn4juKvsakahRGqjxOPytHWspYoBogyhS/V6QSw==; fpk2=0845b309c7b9b957afd9ecf775a4c21f; passport_csrf_token=d80e0c5b2fa2328219856be5ba7e671e; passport_csrf_token_default=d80e0c5b2fa2328219856be5ba7e671e; odin_tt=3c891091d2eb0f4718c1d5645bc4a0017032d4d5aa989decb729e9da2ad570918cbe5e9133dc6b145fa8c758de98efe32ff1f81aa0d611e838cc73ab08ef7d3f6adf66ab4d10e8372ddd628f94f16b8e; volume_info=%7B%22isUserMute%22%3Afalse%2C%22isMute%22%3Afalse%2C%22volume%22%3A0.5%7D; bd_ticket_guard_client_web_domain=2; FORCE_LOGIN=%7B%22videoConsumedRemainSeconds%22%3A180%7D; UIFID=0de8750d2b188f4235dbfd208e44abbb976428f0720eb983255afefa45d39c0c6532e1d4768dd8587bf919f866ff139655a3c2b735923234f371c699560c657923fd3d6c5b63ab7bb9b83423b6cb4787e2ce66a7fbc4ecb24c8570f520fe6de068bbb95115023c0c6c1b6ee31b49fb7e3996fb8349f43a3fd8b7a61cd9e18e8fe65eb6a7c13de4c0960d84e344b644725db3eb2fa6b7caf821de1b50527979f2; is_dash_user=1; biz_trace_id=b57a241f; bd_ticket_guard_client_data=eyJiZC10aWNrZXQtZ3VhcmQtdmVyc2lvbiI6MiwiYmQtdGlja2V0LWd1YXJkLWl0ZXJhdGlvbi12ZXJzaW9uIjoxLCJiZC10aWNrZXQtZ3VhcmQtcmVlLXB1YmxpYy1rZXkiOiJCTEo2R0lDalVoWW1XcHpGOFdrN0Vrc0dXcCtaUzNKY1g4NGNGY2k0TTl1TEowNjdUb21mbFU5aDdvWVBGamhNRWNRQWtKdnN1MnM3RmpTWnlJQXpHMjA9IiwiYmQtdGlja2V0LWd1YXJkLXdlYi12ZXJzaW9uIjoyfQ%3D%3D; download_guide=%221%2F20241216%2F0%22; sdk_source_info=7e276470716a68645a606960273f276364697660272927676c715a6d6069756077273f276364697660272927666d776a68605a607d71606b766c6a6b5a7666776c7571273f275e58272927666a6b766a69605a696c6061273f27636469766027292762696a6764695a7364776c6467696076273f275e5827292771273f273d33323131333c3036313632342778; bit_env=RiOY4jzzpxZoVCl6zdVSVhVRjdwHRTxqcqWdqMBZLPGjMdB4Tax1kAELHNTVAAh72KuhumewE4Lq6f0-VJ2UpJrkrhSxoPw9LUb3zQrq1OSwbeSPHkRlRgRQvO89sItdGUyq1oFr0XyRCnMYG87KSeWyc4x0czGR0o50hTDoDLG5rJVoRcdQOLvjiAegsqyytKF59sPX_QM9qffK2SqYsg0hCggURc_AI6kguDDE5DvG0bnyz1utw4z1eEnIoLrkGDqzqBZj4dOAr0BVU6ofbsS-pOQ2u2PM1dLP9FlBVBlVaqYVgHJeSLsR5k76BRTddUjTb4zEilVIEwAMJWGN4I1BxVt6fC9B5tBQpuT0lj3n3eKXCKXZsd8FrEs5_pbfDsxV-e_WMiXI2ff4qxiTC0U73sfo9OpicKICtZjdq8qsHxJuu6wVR36zvXeL2Wch5C6MzprNvkivv0l8nbh2mSgy1nabZr3dmU6NcR-Bg3Q3xTWUlR9aAUmpopC-cNuXjgLpT-Lw1AYGilSUnCvosth1Gfypq-b0MpgmdSDgTrQ%3D; gulu_source_res=eyJwX2luIjoiMDhjOGQ3ZTJiODQyNjZkZWI5Y2VkMGJiODNlNmY1ZWY0ZjMyNTE2ZmYyZjAzNDMzZjI0OWU1Y2Q1NTczNTk5NyJ9; passport_auth_mix_state=hp9bc3dgb1tm5wd8p82zawus27g0e3ue; IsDouyinActive=false",
|
||||
"cookie": settings.DOUYIN_COOKIE,
|
||||
"user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
|
||||
}
|
||||
|
||||
logger.info(f"[SuperIPAgent] Requesting page with Cookie...")
|
||||
# 必须 verify=False 否则有些环境会报错
|
||||
response = requests.get(target_url, headers=headers_with_cookie, timeout=10)
|
||||
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
response = await client.get(target_url, headers=headers_with_cookie)
|
||||
|
||||
# 4. 解析 RENDER_DATA
|
||||
content_match = re.findall(r'<script id="RENDER_DATA" type="application/json">(.*?)</script>', response.text)
|
||||
@@ -290,17 +299,18 @@ async def download_douyin_manual(url: str, temp_dir: Path, timestamp: int) -> Op
|
||||
|
||||
logger.info(f"[SuperIPAgent] Found video URL: {video_url[:50]}...")
|
||||
|
||||
# 6. 下载 (带 Header)
|
||||
# 6. 下载 (带 Header) - 使用异步 httpx
|
||||
temp_path = temp_dir / f"douyin_manual_{timestamp}.mp4"
|
||||
download_headers = {
|
||||
'Referer': 'https://www.douyin.com/',
|
||||
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36',
|
||||
}
|
||||
|
||||
dl_resp = requests.get(video_url, headers=download_headers, stream=True, timeout=60)
|
||||
async with httpx.AsyncClient(timeout=60.0) as client:
|
||||
async with client.stream("GET", video_url, headers=download_headers) as dl_resp:
|
||||
if dl_resp.status_code == 200:
|
||||
with open(temp_path, 'wb') as f:
|
||||
for chunk in dl_resp.iter_content(chunk_size=1024):
|
||||
async for chunk in dl_resp.aiter_bytes(chunk_size=8192):
|
||||
f.write(chunk)
|
||||
|
||||
logger.info(f"[SuperIPAgent] Downloaded successfully: {temp_path}")
|
||||
|
||||
@@ -27,13 +27,20 @@ async def generate_video(
|
||||
|
||||
|
||||
@router.get("/tasks/{task_id}")
|
||||
async def get_task_status(task_id: str):
|
||||
return success_response(get_task(task_id))
|
||||
async def get_task_status(task_id: str, current_user: dict = Depends(get_current_user)):
|
||||
task = get_task(task_id)
|
||||
# 验证任务归属：只能查看自己的任务
|
||||
if task.get("status") != "not_found" and task.get("user_id") != current_user["id"]:
|
||||
return success_response({"status": "not_found"})
|
||||
return success_response(task)
|
||||
|
||||
|
||||
@router.get("/tasks")
|
||||
async def list_tasks_view():
|
||||
return success_response({"tasks": list_tasks()})
|
||||
async def list_tasks_view(current_user: dict = Depends(get_current_user)):
|
||||
# 只返回当前用户的任务
|
||||
all_tasks = list_tasks()
|
||||
user_tasks = [t for t in all_tasks if t.get("user_id") == current_user["id"]]
|
||||
return success_response({"tasks": user_tasks})
|
||||
|
||||
|
||||
@router.get("/lipsync/health")
|
||||
|
||||
@@ -277,12 +277,10 @@ async def process_video_generation(task_id: str, req: GenerateRequest, user_id:
|
||||
_update_task(task_id, message="正在上传结果...", progress=95)
|
||||
|
||||
storage_path = f"{user_id}/{task_id}_output.mp4"
|
||||
with open(final_output_local_path, "rb") as f:
|
||||
file_data = f.read()
|
||||
await storage_service.upload_file(
|
||||
await storage_service.upload_file_from_path(
|
||||
bucket=storage_service.BUCKET_OUTPUTS,
|
||||
path=storage_path,
|
||||
file_data=file_data,
|
||||
storage_path=storage_path,
|
||||
local_file_path=str(final_output_local_path),
|
||||
content_type="video/mp4"
|
||||
)
|
||||
|
||||
|
||||
@@ -51,7 +51,10 @@ class GLMService:
|
||||
client = self._get_client()
|
||||
logger.info(f"Calling GLM API with model: {settings.GLM_MODEL}")
|
||||
|
||||
response = client.chat.completions.create(
|
||||
# 使用 asyncio.to_thread 包装同步 SDK 调用，避免阻塞事件循环
|
||||
import asyncio
|
||||
response = await asyncio.to_thread(
|
||||
client.chat.completions.create,
|
||||
model=settings.GLM_MODEL,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
thinking={"type": "disabled"}, # 禁用思考模式，加快响应
|
||||
@@ -96,7 +99,10 @@ class GLMService:
|
||||
client = self._get_client()
|
||||
logger.info(f"Using GLM to rewrite script")
|
||||
|
||||
response = client.chat.completions.create(
|
||||
# 使用 asyncio.to_thread 包装同步 SDK 调用，避免阻塞事件循环
|
||||
import asyncio
|
||||
response = await asyncio.to_thread(
|
||||
client.chat.completions.create,
|
||||
model=settings.GLM_MODEL,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
thinking={"type": "disabled"},
|
||||
|
||||
@@ -398,18 +398,23 @@ class LipSyncService:
|
||||
raise e
|
||||
|
||||
async def _local_generate_subprocess(self, video_path: str, audio_path: str, output_path: str) -> str:
|
||||
"""原有的 subprocess 逻辑提取为独立方法"""
|
||||
logger.info("🔄 调用 LatentSync 推理 (subprocess)...")
|
||||
# ... (此处仅为占位符提示，实际代码需要调整结构以避免重复，
|
||||
# 但鉴于原有 _local_generate 的结构，最简单的方法是在 _local_generate 内部做判断，
|
||||
# 如果 use_server 失败，可以 retry 或者 _local_generate 不做拆分，直接在里面写逻辑)
|
||||
# 为了最小化改动且保持安全，上面的 _call_persistent_server 如果失败，
|
||||
# 最好不要自动回退（可能导致双重资源消耗），而是直接报错让用户检查服务。
|
||||
# 但为了用户体验，我们可以允许回退。
|
||||
# *修正策略*:
|
||||
# 我将不拆分 _local_generate_subprocess，而是将 subprocess 逻辑保留在 _local_generate 的后半部分。
|
||||
# 如果 self.use_server 为 True，先尝试调用 server，成功则 return，失败则继续往下走。
|
||||
pass
|
||||
"""
|
||||
原有的 subprocess 回退逻辑
|
||||
|
||||
注意：subprocess 回退已被禁用，原因如下：
|
||||
1. subprocess 模式需要重新加载模型，消耗大量时间和显存
|
||||
2. 如果常驻服务不可用，应该让用户知道并修复服务，而非静默回退
|
||||
3. 避免双重资源消耗导致的 GPU OOM
|
||||
|
||||
如果常驻服务不可用，请检查：
|
||||
- 服务是否启动: python scripts/server.py (在 models/LatentSync 目录)
|
||||
- 端口是否被占用: lsof -i:8007
|
||||
- GPU 显存是否充足: nvidia-smi
|
||||
"""
|
||||
raise RuntimeError(
|
||||
"LatentSync 常驻服务不可用，无法进行唇形同步。"
|
||||
"请确保 LatentSync 服务已启动 (cd models/LatentSync && python scripts/server.py)"
|
||||
)
|
||||
|
||||
async def _remote_generate(
|
||||
self,
|
||||
|
||||
@@ -52,13 +52,21 @@ class RemotionService:
|
||||
输出视频路径
|
||||
"""
|
||||
# 构建命令参数
|
||||
cmd = [
|
||||
"npx", "ts-node", "render.ts",
|
||||
# 优先使用预编译的 JS 文件（更快），如果不存在则回退到 ts-node
|
||||
compiled_js = self.remotion_dir / "dist" / "render.js"
|
||||
if compiled_js.exists():
|
||||
cmd = ["node", "dist/render.js"]
|
||||
logger.info("Using pre-compiled render.js for faster startup")
|
||||
else:
|
||||
cmd = ["npx", "ts-node", "render.ts"]
|
||||
logger.warning("Using ts-node (slower). Run 'npm run build:render' to compile for faster startup.")
|
||||
|
||||
cmd.extend([
|
||||
"--video", str(video_path),
|
||||
"--output", str(output_path),
|
||||
"--fps", str(fps),
|
||||
"--enableSubtitles", str(enable_subtitles).lower()
|
||||
]
|
||||
])
|
||||
|
||||
if captions_path:
|
||||
cmd.extend(["--captions", str(captions_path)])
|
||||
|
||||
@@ -7,9 +7,12 @@ from pathlib import Path
|
||||
import asyncio
|
||||
import functools
|
||||
import os
|
||||
import shutil
|
||||
|
||||
# Supabase Storage 本地存储根目录
|
||||
SUPABASE_STORAGE_LOCAL_PATH = Path("/home/rongye/ProgramFiles/Supabase/volumes/storage/stub/stub")
|
||||
# Supabase Storage 本地存储根目录（从环境变量读取，支持不同部署环境）
|
||||
SUPABASE_STORAGE_LOCAL_PATH = Path(
|
||||
os.getenv("SUPABASE_STORAGE_LOCAL_PATH", "/home/rongye/ProgramFiles/Supabase/volumes/storage/stub/stub")
|
||||
)
|
||||
|
||||
class StorageService:
|
||||
def __init__(self):
|
||||
@@ -100,6 +103,45 @@ class StorageService:
|
||||
logger.error(f"Storage upload failed: {e}")
|
||||
raise e
|
||||
|
||||
async def upload_file_from_path(self, bucket: str, storage_path: str, local_file_path: str, content_type: str) -> str:
|
||||
"""
|
||||
从本地文件路径上传文件到 Supabase Storage
|
||||
|
||||
使用分块读取减少内存峰值，避免大文件整读入内存
|
||||
|
||||
Args:
|
||||
bucket: 存储桶名称
|
||||
storage_path: Storage 中的目标路径
|
||||
local_file_path: 本地文件的绝对路径
|
||||
content_type: MIME 类型
|
||||
"""
|
||||
local_file = Path(local_file_path)
|
||||
if not local_file.exists():
|
||||
raise FileNotFoundError(f"本地文件不存在: {local_file_path}")
|
||||
|
||||
loop = asyncio.get_running_loop()
|
||||
file_size = local_file.stat().st_size
|
||||
|
||||
# 分块读取文件，避免大文件整读入内存
|
||||
# 虽然最终还是需要拼接成 bytes 传给 SDK，但分块读取可以减少 IO 压力
|
||||
def read_file_chunked():
|
||||
chunks = []
|
||||
chunk_size = 10 * 1024 * 1024 # 10MB per chunk
|
||||
with open(local_file_path, "rb") as f:
|
||||
while True:
|
||||
chunk = f.read(chunk_size)
|
||||
if not chunk:
|
||||
break
|
||||
chunks.append(chunk)
|
||||
return b"".join(chunks)
|
||||
|
||||
if file_size > 50 * 1024 * 1024: # 大于 50MB 记录日志
|
||||
logger.info(f"大文件上传: {file_size / 1024 / 1024:.1f}MB")
|
||||
|
||||
file_data = await loop.run_in_executor(None, read_file_chunked)
|
||||
|
||||
return await self.upload_file(bucket, storage_path, file_data, content_type)
|
||||
|
||||
async def get_signed_url(self, bucket: str, path: str, expires_in: int = 3600) -> str:
|
||||
"""异步获取签名访问链接"""
|
||||
try:
|
||||
|
||||
@@ -35,11 +35,16 @@ class VideoService:
|
||||
|
||||
def _get_duration(self, file_path: str) -> float:
|
||||
# Synchronous call for BackgroundTasks compatibility
|
||||
cmd = f'ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 "{file_path}"'
|
||||
# 使用参数列表形式避免 shell=True 的命令注入风险
|
||||
cmd = [
|
||||
'ffprobe', '-v', 'error',
|
||||
'-show_entries', 'format=duration',
|
||||
'-of', 'default=noprint_wrappers=1:nokey=1',
|
||||
file_path
|
||||
]
|
||||
try:
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
shell=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
@@ -3,15 +3,8 @@
|
||||
import { createContext, useContext, useState, useEffect, ReactNode } from "react";
|
||||
import api from "@/shared/api/axios";
|
||||
import { ApiResponse, unwrap } from "@/shared/api/types";
|
||||
import { User } from "@/shared/types/user";
|
||||
|
||||
interface User {
|
||||
id: string;
|
||||
phone: string;
|
||||
username: string | null;
|
||||
role: string;
|
||||
is_active: boolean;
|
||||
expires_at: string | null;
|
||||
}
|
||||
|
||||
interface AuthContextType {
|
||||
userId: string | null;
|
||||
|
||||
@@ -1,20 +1,15 @@
|
||||
/**
|
||||
* 认证工具函数
|
||||
*/
|
||||
import { User } from "@/shared/types/user";
|
||||
|
||||
// Re-export User 类型以保持向后兼容
|
||||
export type { User };
|
||||
|
||||
const API_BASE = typeof window === 'undefined'
|
||||
? (process.env.NEXT_PUBLIC_API_URL || 'http://localhost:8006')
|
||||
: '';
|
||||
|
||||
export interface User {
|
||||
id: string;
|
||||
phone: string;
|
||||
username: string | null;
|
||||
role: string;
|
||||
is_active: boolean;
|
||||
expires_at: string | null;
|
||||
}
|
||||
|
||||
export interface AuthResponse {
|
||||
success: boolean;
|
||||
message: string;
|
||||
|
||||
13
frontend/src/shared/types/user.ts
Normal file
13
frontend/src/shared/types/user.ts
Normal file
@@ -0,0 +1,13 @@
|
||||
/**
|
||||
* 用户类型定义
|
||||
* 统一管理用户相关类型，避免重复定义
|
||||
*/
|
||||
|
||||
export interface User {
|
||||
id: string;
|
||||
phone: string;
|
||||
username: string | null;
|
||||
role: string;
|
||||
is_active: boolean;
|
||||
expires_at: string | null;
|
||||
}
|
||||
@@ -65,14 +65,15 @@ async def lifespan(app: FastAPI):
|
||||
# --- 模型加载逻辑 (参考 inference.py) ---
|
||||
print("⏳ 正在加载 LatentSync 模型...")
|
||||
|
||||
# 默认配置路径 (相对于根目录)
|
||||
unet_config_path = "configs/unet/stage2_512.yaml"
|
||||
ckpt_path = "checkpoints/latentsync_unet.pt"
|
||||
# 使用绝对路径，确保可以从任意目录启动
|
||||
latentsync_root = Path(__file__).resolve().parent.parent # scripts -> LatentSync 根目录
|
||||
unet_config_path = latentsync_root / "configs" / "unet" / "stage2_512.yaml"
|
||||
ckpt_path = latentsync_root / "checkpoints" / "latentsync_unet.pt"
|
||||
|
||||
if not os.path.exists(unet_config_path):
|
||||
print(f"⚠️ 找不到配置文件: {unet_config_path}，请确保在 models/LatentSync 根目录运行")
|
||||
if not unet_config_path.exists():
|
||||
print(f"⚠️ 找不到配置文件: {unet_config_path}")
|
||||
|
||||
config = OmegaConf.load(unet_config_path)
|
||||
config = OmegaConf.load(str(unet_config_path))
|
||||
|
||||
# Check GPU
|
||||
is_fp16_supported = torch.cuda.is_available() and torch.cuda.get_device_capability()[0] > 7
|
||||
@@ -85,13 +86,13 @@ async def lifespan(app: FastAPI):
|
||||
else:
|
||||
print("⚠️ 警告: 未检测到 GPU，将使用 CPU 进行推理 (速度极慢)")
|
||||
|
||||
scheduler = DDIMScheduler.from_pretrained("configs")
|
||||
scheduler = DDIMScheduler.from_pretrained(str(latentsync_root / "configs"))
|
||||
|
||||
# Whisper Model
|
||||
if config.model.cross_attention_dim == 768:
|
||||
whisper_path = "checkpoints/whisper/small.pt"
|
||||
whisper_path = str(latentsync_root / "checkpoints" / "whisper" / "small.pt")
|
||||
else:
|
||||
whisper_path = "checkpoints/whisper/tiny.pt"
|
||||
whisper_path = str(latentsync_root / "checkpoints" / "whisper" / "tiny.pt")
|
||||
|
||||
audio_encoder = Audio2Feature(
|
||||
model_path=whisper_path,
|
||||
@@ -108,7 +109,7 @@ async def lifespan(app: FastAPI):
|
||||
# UNet
|
||||
unet, _ = UNet3DConditionModel.from_pretrained(
|
||||
OmegaConf.to_container(config.model),
|
||||
ckpt_path,
|
||||
str(ckpt_path),
|
||||
device="cpu", # Load to CPU first to save memory during init
|
||||
)
|
||||
unet = unet.to(dtype=dtype)
|
||||
@@ -129,6 +130,7 @@ async def lifespan(app: FastAPI):
|
||||
models["pipeline"] = pipeline
|
||||
models["config"] = config
|
||||
models["dtype"] = dtype
|
||||
models["latentsync_root"] = latentsync_root
|
||||
|
||||
print("✅ LatentSync 模型加载完成，服务就绪！")
|
||||
yield
|
||||
@@ -167,6 +169,7 @@ async def generate_lipsync(req: LipSyncRequest):
|
||||
pipeline = models["pipeline"]
|
||||
config = models["config"]
|
||||
dtype = models["dtype"]
|
||||
latentsync_root = models["latentsync_root"]
|
||||
|
||||
# Set seed
|
||||
if req.seed != -1:
|
||||
@@ -185,7 +188,7 @@ async def generate_lipsync(req: LipSyncRequest):
|
||||
weight_dtype=dtype,
|
||||
width=config.data.resolution,
|
||||
height=config.data.resolution,
|
||||
mask_image_path=config.data.mask_image_path,
|
||||
mask_image_path=str(latentsync_root / config.data.mask_image_path),
|
||||
temp_dir=req.temp_dir,
|
||||
)
|
||||
|
||||
|
||||
@@ -5,7 +5,9 @@
|
||||
"scripts": {
|
||||
"start": "remotion studio",
|
||||
"build": "remotion bundle",
|
||||
"render": "npx ts-node render.ts"
|
||||
"build:render": "npx tsc render.ts --outDir dist --esModuleInterop --skipLibCheck",
|
||||
"render": "npx ts-node render.ts",
|
||||
"render:fast": "node dist/render.js"
|
||||
},
|
||||
"dependencies": {
|
||||
"remotion": "^4.0.0",
|
||||
|
||||
@@ -16,6 +16,8 @@ interface RenderOptions {
|
||||
captionsPath?: string;
|
||||
title?: string;
|
||||
titleDuration?: number;
|
||||
subtitleStyle?: Record<string, unknown>;
|
||||
titleStyle?: Record<string, unknown>;
|
||||
outputPath: string;
|
||||
fps?: number;
|
||||
enableSubtitles?: boolean;
|
||||
@@ -53,6 +55,20 @@ async function parseArgs(): Promise<RenderOptions> {
|
||||
case 'enableSubtitles':
|
||||
options.enableSubtitles = value === 'true';
|
||||
break;
|
||||
case 'subtitleStyle':
|
||||
try {
|
||||
options.subtitleStyle = JSON.parse(value);
|
||||
} catch (e) {
|
||||
console.warn('Invalid subtitleStyle JSON');
|
||||
}
|
||||
break;
|
||||
case 'titleStyle':
|
||||
try {
|
||||
options.titleStyle = JSON.parse(value);
|
||||
} catch (e) {
|
||||
console.warn('Invalid titleStyle JSON');
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -84,20 +100,22 @@ async function main() {
|
||||
let videoWidth = 1280;
|
||||
let videoHeight = 720;
|
||||
try {
|
||||
// 使用 ffprobe 获取视频时长
|
||||
const { execSync } = require('child_process');
|
||||
const ffprobeOutput = execSync(
|
||||
`ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 "${options.videoPath}"`,
|
||||
{ encoding: 'utf-8' }
|
||||
// 使用 promisified exec 异步获取视频信息，避免阻塞主线程
|
||||
const { promisify } = require('util');
|
||||
const { exec } = require('child_process');
|
||||
const execAsync = promisify(exec);
|
||||
|
||||
// 获取视频时长
|
||||
const { stdout: durationOutput } = await execAsync(
|
||||
`ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 "${options.videoPath}"`
|
||||
);
|
||||
const durationInSeconds = parseFloat(ffprobeOutput.trim());
|
||||
const durationInSeconds = parseFloat(durationOutput.trim());
|
||||
durationInFrames = Math.ceil(durationInSeconds * fps);
|
||||
console.log(`Video duration: ${durationInSeconds}s (${durationInFrames} frames at ${fps}fps)`);
|
||||
|
||||
// 使用 ffprobe 获取视频尺寸
|
||||
const dimensionsOutput = execSync(
|
||||
`ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of csv=s=x:p=0 "${options.videoPath}"`,
|
||||
{ encoding: 'utf-8' }
|
||||
// 获取视频尺寸
|
||||
const { stdout: dimensionsOutput } = await execAsync(
|
||||
`ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of csv=s=x:p=0 "${options.videoPath}"`
|
||||
);
|
||||
const [width, height] = dimensionsOutput.trim().split('x').map(Number);
|
||||
if (width && height) {
|
||||
@@ -131,6 +149,8 @@ async function main() {
|
||||
captions,
|
||||
title: options.title,
|
||||
titleDuration: options.titleDuration || 3,
|
||||
subtitleStyle: options.subtitleStyle,
|
||||
titleStyle: options.titleStyle,
|
||||
enableSubtitles: options.enableSubtitles !== false,
|
||||
},
|
||||
});
|
||||
@@ -153,6 +173,8 @@ async function main() {
|
||||
captions,
|
||||
title: options.title,
|
||||
titleDuration: options.titleDuration || 3,
|
||||
subtitleStyle: options.subtitleStyle,
|
||||
titleStyle: options.titleStyle,
|
||||
enableSubtitles: options.enableSubtitles !== false,
|
||||
},
|
||||
onProgress: ({ progress }) => {
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import React from 'react';
|
||||
import { AbsoluteFill, Composition } from 'remotion';
|
||||
import { VideoLayer } from './components/VideoLayer';
|
||||
import { Title } from './components/Title';
|
||||
import { Subtitles } from './components/Subtitles';
|
||||
import { Title, TitleStyle } from './components/Title';
|
||||
import { Subtitles, SubtitleStyle } from './components/Subtitles';
|
||||
import { CaptionsData } from './utils/captions';
|
||||
|
||||
export interface VideoProps {
|
||||
@@ -12,6 +12,8 @@ export interface VideoProps {
|
||||
title?: string;
|
||||
titleDuration?: number;
|
||||
enableSubtitles?: boolean;
|
||||
subtitleStyle?: SubtitleStyle;
|
||||
titleStyle?: TitleStyle;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -25,6 +27,8 @@ export const Video: React.FC<VideoProps> = ({
|
||||
title,
|
||||
titleDuration = 3,
|
||||
enableSubtitles = true,
|
||||
subtitleStyle,
|
||||
titleStyle,
|
||||
}) => {
|
||||
return (
|
||||
<AbsoluteFill style={{ backgroundColor: 'black' }}>
|
||||
@@ -33,12 +37,12 @@ export const Video: React.FC<VideoProps> = ({
|
||||
|
||||
{/* 中层：字幕 */}
|
||||
{enableSubtitles && captions && (
|
||||
<Subtitles captions={captions} />
|
||||
<Subtitles captions={captions} style={subtitleStyle} />
|
||||
)}
|
||||
|
||||
{/* 顶层：标题 */}
|
||||
{title && (
|
||||
<Title title={title} duration={titleDuration} />
|
||||
<Title title={title} duration={titleDuration} style={titleStyle} />
|
||||
)}
|
||||
</AbsoluteFill>
|
||||
);
|
||||
|
||||
@@ -1,28 +1,59 @@
|
||||
import React from 'react';
|
||||
import { AbsoluteFill, useCurrentFrame, useVideoConfig } from 'remotion';
|
||||
import { AbsoluteFill, useCurrentFrame, useVideoConfig, staticFile } from 'remotion';
|
||||
import {
|
||||
CaptionsData,
|
||||
getCurrentSegment,
|
||||
getCurrentWordIndex,
|
||||
} from '../utils/captions';
|
||||
|
||||
export interface SubtitleStyle {
|
||||
font_file?: string;
|
||||
fontFamily?: string;
|
||||
font_family?: string;
|
||||
fontSize?: number;
|
||||
font_size?: number;
|
||||
highlightColor?: string;
|
||||
highlight_color?: string;
|
||||
normalColor?: string;
|
||||
normal_color?: string;
|
||||
strokeColor?: string;
|
||||
stroke_color?: string;
|
||||
strokeSize?: number;
|
||||
stroke_size?: number;
|
||||
letterSpacing?: number;
|
||||
letter_spacing?: number;
|
||||
bottomMargin?: number;
|
||||
bottom_margin?: number;
|
||||
}
|
||||
|
||||
interface SubtitlesProps {
|
||||
captions: CaptionsData;
|
||||
highlightColor?: string;
|
||||
normalColor?: string;
|
||||
fontSize?: number;
|
||||
style?: SubtitleStyle;
|
||||
}
|
||||
|
||||
/**
|
||||
* 逐字高亮字幕组件
|
||||
* 根据时间戳逐字高亮显示字幕（无背景，纯文字描边）
|
||||
*/
|
||||
export const Subtitles: React.FC<SubtitlesProps> = ({
|
||||
captions,
|
||||
highlightColor = '#FFFF00',
|
||||
normalColor = '#FFFFFF',
|
||||
fontSize = 52,
|
||||
}) => {
|
||||
const getFontFormat = (fontFile?: string) => {
|
||||
if (!fontFile) return 'truetype';
|
||||
const ext = fontFile.split('.').pop()?.toLowerCase();
|
||||
if (ext === 'otf') return 'opentype';
|
||||
return 'truetype';
|
||||
};
|
||||
|
||||
const buildTextShadow = (color: string, size: number) => {
|
||||
return [
|
||||
`-${size}px -${size}px 0 ${color}`,
|
||||
`${size}px -${size}px 0 ${color}`,
|
||||
`-${size}px ${size}px 0 ${color}`,
|
||||
`${size}px ${size}px 0 ${color}`,
|
||||
`0 0 ${size * 4}px rgba(0,0,0,0.9)`,
|
||||
`0 4px 8px rgba(0,0,0,0.6)`
|
||||
].join(',');
|
||||
};
|
||||
|
||||
export const Subtitles: React.FC<SubtitlesProps> = ({ captions, style }) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps } = useVideoConfig();
|
||||
|
||||
@@ -38,25 +69,49 @@ export const Subtitles: React.FC<SubtitlesProps> = ({
|
||||
// 获取当前高亮字的索引
|
||||
const currentWordIndex = getCurrentWordIndex(currentSegment, currentTimeInSeconds);
|
||||
|
||||
const fontFile = style?.font_file;
|
||||
const fontFamily = style?.fontFamily || style?.font_family;
|
||||
const fontSize = style?.fontSize || style?.font_size || 52;
|
||||
const highlightColor = style?.highlightColor || style?.highlight_color || '#FFFF00';
|
||||
const normalColor = style?.normalColor || style?.normal_color || '#FFFFFF';
|
||||
const strokeColor = style?.strokeColor || style?.stroke_color || '#000000';
|
||||
const strokeSize = style?.strokeSize || style?.stroke_size || 3;
|
||||
const letterSpacing = style?.letterSpacing || style?.letter_spacing || 2;
|
||||
const bottomMargin = style?.bottomMargin || style?.bottom_margin;
|
||||
const fontFamilyName = fontFamily || 'SubtitleFont';
|
||||
const fontFamilyCss = fontFile
|
||||
? `'${fontFamilyName}'`
|
||||
: '"PingFang SC", "Hiragino Sans GB", "Microsoft YaHei", "Noto Sans SC", sans-serif';
|
||||
|
||||
return (
|
||||
<AbsoluteFill
|
||||
style={{
|
||||
justifyContent: 'flex-end',
|
||||
alignItems: 'center',
|
||||
paddingBottom: '6%',
|
||||
paddingBottom: typeof bottomMargin === 'number' ? `${bottomMargin}px` : '6%',
|
||||
}}
|
||||
>
|
||||
{fontFile && (
|
||||
<style>{`
|
||||
@font-face {
|
||||
font-family: '${fontFamilyName}';
|
||||
src: url('${staticFile(fontFile)}') format('${getFontFormat(fontFile)}');
|
||||
font-weight: 400;
|
||||
font-style: normal;
|
||||
}
|
||||
`}</style>
|
||||
)}
|
||||
<p
|
||||
style={{
|
||||
margin: 0,
|
||||
fontSize: `${fontSize}px`,
|
||||
fontFamily: '"PingFang SC", "Hiragino Sans GB", "Microsoft YaHei", "Noto Sans SC", sans-serif',
|
||||
fontFamily: fontFamilyCss,
|
||||
fontWeight: 800,
|
||||
lineHeight: 1.4,
|
||||
textAlign: 'center',
|
||||
maxWidth: '90%',
|
||||
wordBreak: 'keep-all',
|
||||
letterSpacing: '2px',
|
||||
letterSpacing: `${letterSpacing}px`,
|
||||
}}
|
||||
>
|
||||
{currentSegment.words.map((word, index) => {
|
||||
@@ -66,14 +121,7 @@ export const Subtitles: React.FC<SubtitlesProps> = ({
|
||||
key={`${word.word}-${index}`}
|
||||
style={{
|
||||
color: isHighlighted ? highlightColor : normalColor,
|
||||
textShadow: `
|
||||
-3px -3px 0 #000,
|
||||
3px -3px 0 #000,
|
||||
-3px 3px 0 #000,
|
||||
3px 3px 0 #000,
|
||||
0 0 12px rgba(0,0,0,0.9),
|
||||
0 4px 8px rgba(0,0,0,0.6)
|
||||
`,
|
||||
textShadow: buildTextShadow(strokeColor, strokeSize),
|
||||
transition: 'color 0.05s ease',
|
||||
}}
|
||||
>
|
||||
|
||||
@@ -4,22 +4,62 @@ import {
|
||||
interpolate,
|
||||
useCurrentFrame,
|
||||
useVideoConfig,
|
||||
staticFile,
|
||||
} from 'remotion';
|
||||
|
||||
export interface TitleStyle {
|
||||
font_file?: string;
|
||||
fontFamily?: string;
|
||||
font_family?: string;
|
||||
fontSize?: number;
|
||||
font_size?: number;
|
||||
color?: string;
|
||||
strokeColor?: string;
|
||||
stroke_color?: string;
|
||||
strokeSize?: number;
|
||||
stroke_size?: number;
|
||||
letterSpacing?: number;
|
||||
letter_spacing?: number;
|
||||
topMargin?: number;
|
||||
top_margin?: number;
|
||||
fontWeight?: number;
|
||||
font_weight?: number;
|
||||
}
|
||||
|
||||
interface TitleProps {
|
||||
title: string;
|
||||
duration?: number; // 标题显示时长（秒）
|
||||
fadeOutStart?: number; // 开始淡出的时间（秒）
|
||||
style?: TitleStyle;
|
||||
}
|
||||
|
||||
/**
|
||||
* 片头标题组件
|
||||
* 在视频顶部显示标题，带淡入淡出效果
|
||||
*/
|
||||
const getFontFormat = (fontFile?: string) => {
|
||||
if (!fontFile) return 'truetype';
|
||||
const ext = fontFile.split('.').pop()?.toLowerCase();
|
||||
if (ext === 'otf') return 'opentype';
|
||||
return 'truetype';
|
||||
};
|
||||
|
||||
const buildTextShadow = (color: string, size: number) => {
|
||||
return [
|
||||
`-${size}px -${size}px 0 ${color}`,
|
||||
`${size}px -${size}px 0 ${color}`,
|
||||
`-${size}px ${size}px 0 ${color}`,
|
||||
`${size}px ${size}px 0 ${color}`,
|
||||
`0 0 ${size * 2}px rgba(0,0,0,0.7)`,
|
||||
`0 4px 8px rgba(0,0,0,0.6)`
|
||||
].join(',');
|
||||
};
|
||||
|
||||
export const Title: React.FC<TitleProps> = ({
|
||||
title,
|
||||
duration = 3,
|
||||
fadeOutStart = 2,
|
||||
style,
|
||||
}) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps } = useVideoConfig();
|
||||
@@ -57,33 +97,52 @@ export const Title: React.FC<TitleProps> = ({
|
||||
{ extrapolateRight: 'clamp' }
|
||||
);
|
||||
|
||||
const fontFile = style?.font_file;
|
||||
const fontFamily = style?.fontFamily || style?.font_family;
|
||||
const fontSize = style?.fontSize || style?.font_size || 72;
|
||||
const color = style?.color || '#FFFFFF';
|
||||
const strokeColor = style?.strokeColor || style?.stroke_color || '#000000';
|
||||
const strokeSize = style?.strokeSize || style?.stroke_size || 8;
|
||||
const letterSpacing = style?.letterSpacing || style?.letter_spacing || 4;
|
||||
const topMargin = style?.topMargin || style?.top_margin;
|
||||
const fontWeight = style?.fontWeight || style?.font_weight || 900;
|
||||
const fontFamilyName = fontFamily || 'TitleFont';
|
||||
const fontFamilyCss = fontFile
|
||||
? `'${fontFamilyName}'`
|
||||
: '"PingFang SC", "Hiragino Sans GB", "Microsoft YaHei", "Noto Sans SC", sans-serif';
|
||||
|
||||
return (
|
||||
<AbsoluteFill
|
||||
style={{
|
||||
justifyContent: 'flex-start',
|
||||
alignItems: 'center',
|
||||
paddingTop: '6%',
|
||||
paddingTop: typeof topMargin === 'number' ? `${topMargin}px` : '6%',
|
||||
opacity,
|
||||
}}
|
||||
>
|
||||
{fontFile && (
|
||||
<style>{`
|
||||
@font-face {
|
||||
font-family: '${fontFamilyName}';
|
||||
src: url('${staticFile(fontFile)}') format('${getFontFormat(fontFile)}');
|
||||
font-weight: 400;
|
||||
font-style: normal;
|
||||
}
|
||||
`}</style>
|
||||
)}
|
||||
<h1
|
||||
style={{
|
||||
transform: `translateY(${translateY}px)`,
|
||||
textAlign: 'center',
|
||||
color: '#FFFFFF',
|
||||
fontSize: '72px',
|
||||
fontWeight: 900,
|
||||
fontFamily: '"PingFang SC", "Hiragino Sans GB", "Microsoft YaHei", "Noto Sans SC", sans-serif',
|
||||
textShadow: `
|
||||
0 0 10px rgba(0,0,0,0.9),
|
||||
0 0 20px rgba(0,0,0,0.7),
|
||||
0 4px 8px rgba(0,0,0,0.8),
|
||||
0 8px 16px rgba(0,0,0,0.5)
|
||||
`,
|
||||
color,
|
||||
fontSize: `${fontSize}px`,
|
||||
fontWeight,
|
||||
fontFamily: fontFamilyCss,
|
||||
textShadow: buildTextShadow(strokeColor, strokeSize),
|
||||
margin: 0,
|
||||
padding: '0 5%',
|
||||
lineHeight: 1.3,
|
||||
letterSpacing: '4px',
|
||||
letterSpacing: `${letterSpacing}px`,
|
||||
}}
|
||||
>
|
||||
{title}
|
||||
|
||||
Reference in New Issue
Block a user