Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
f879fb0001 | ||
|
|
b289006844 |
@@ -161,6 +161,8 @@ backend/user_data/{user_uuid}/cookies/
|
||||
- 业务逻辑写在 service/workflow。
|
||||
- 数据库访问写在 repositories。
|
||||
- 统一使用 `loguru` 打日志。
|
||||
- GLM SDK 调用统一收口到 `services/glm_service.py`（通过统一入口方法），避免在模块内重复拼装 `chat.completions.create` 调用代码。
|
||||
- 涉及文案深度学习的抓取调用，router 侧应透传 `current_user.id` 到 `creator_scraper`，以便复用用户 Cookie 上下文并保持 `analysis_id` 用户隔离。
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -117,6 +117,13 @@ backend/
|
||||
|
||||
9. **工具 (Tools)**
|
||||
* `POST /api/tools/extract-script`: 从视频链接提取文案（需登录）
|
||||
* `POST /api/tools/analyze-creator`: 分析博主标题并返回热门话题（需登录）
|
||||
* `POST /api/tools/generate-topic-script`: 基于选中话题生成文案（需登录）
|
||||
|
||||
> 文案深度学习说明：
|
||||
> - 平台支持：抖音 / B站博主主页链接。
|
||||
> - 抓取策略：当前统一使用 Playwright 主链路抓取标题（抖音/B站），并结合用户登录态 Cookie 上下文增强成功率。
|
||||
> - `analysis_id` 绑定 `user_id` 且有 TTL（默认 20 分钟），用于后续“生成文案”阶段安全读取标题上下文。
|
||||
|
||||
10. **健康检查**
|
||||
* `GET /api/videos/lipsync/health`: 唇形同步服务健康状态（含 LatentSync + MuseTalk + 混合路由阈值）
|
||||
@@ -241,7 +248,7 @@ pip install -r requirements.txt
|
||||
SUPABASE_URL=http://localhost:8008
|
||||
SUPABASE_KEY=your_service_role_key
|
||||
|
||||
# GLM API (用于 AI 标题生成)
|
||||
# GLM API (用于 AI 标题/改写/翻译/文案深度学习)
|
||||
GLM_API_KEY=your_glm_api_key
|
||||
|
||||
# LatentSync 配置
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
1. 修复首页与发布成功弹窗点击下载时被浏览器当作在线播放（新开标签页）的问题。
|
||||
2. 将下载修复开始后的开发内容从 `Day31` 拆分到 `Day32`，保持日志按天清晰归档。
|
||||
3. 根据安全审计报告（`Temp/安全审计报告.md`），实施第一批 6 项无功能风险的安全修复。
|
||||
4. 统一弹窗关闭交互：默认支持点空白关闭，发布成功清理弹窗保持强制留存。
|
||||
4. 统一弹窗关闭交互（仅关闭策略）：默认支持点空白关闭，发布成功清理弹窗保持强制留存。
|
||||
|
||||
---
|
||||
|
||||
@@ -91,12 +91,13 @@
|
||||
|
||||
---
|
||||
|
||||
## ✅ 4) 弹窗关闭交互统一（UX）
|
||||
## ✅ 4) 弹窗关闭策略统一（UX）
|
||||
|
||||
### 目标
|
||||
|
||||
- 保持统一交互预期：业务弹窗默认可通过 `X` 与点击遮罩关闭。
|
||||
- 保留关键流程保护：发布成功清理弹窗继续禁止遮罩关闭，避免误触导致流程中断。
|
||||
- 说明：按钮位置与视觉样式统一属于 Day33 范畴，本日志仅记录关闭策略统一。
|
||||
|
||||
### 调整内容
|
||||
|
||||
@@ -126,7 +127,7 @@
|
||||
| `frontend/src/components/AccountSettingsDropdown.tsx` | 修改密码弹窗支持点击遮罩关闭 |
|
||||
| `frontend/src/features/home/ui/RefAudioPanel.tsx` | 录音弹窗改为 `closeOnOverlay={!isRecording}`（录音中禁遮罩关闭） |
|
||||
| `Docs/DevLogs/Day31.md` | 移除下载修复章节与对应验证/覆盖项（迁入 Day32） |
|
||||
| `Docs/TASK_COMPLETE.md` | 新增 Day32 Current 区块，Day31 取消 Current |
|
||||
| `Docs/TASK_COMPLETE.md` | 当日新增 Day32 区块并接棒 Current（后续由 Day33 接棒 Current） |
|
||||
| `Docs/BACKEND_README.md` | 补充 `/api/videos/generated/{video_id}/download` 接口说明 |
|
||||
| `Docs/BACKEND_DEV.md` | 补充下载接口 `attachment` 约定 |
|
||||
| `Docs/FRONTEND_README.md` | 补充首页/发布弹窗下载统一同源接口说明 |
|
||||
|
||||
290
Docs/DevLogs/Day33.md
Normal file
290
Docs/DevLogs/Day33.md
Normal file
@@ -0,0 +1,290 @@
|
||||
## 抖音短链文案提取稳健性修复 (Day 33)
|
||||
|
||||
### 概述
|
||||
|
||||
今天聚焦修复「文案提取助手」里抖音分享短链/口令文本偶发提取失败的问题，并补齐多种抖音落地 URL 形态的兼容。
|
||||
|
||||
---
|
||||
|
||||
## ✅ 1) 问题复盘
|
||||
|
||||
### 现象
|
||||
|
||||
- 复制抖音分享口令文本（含 `v.douyin.com` 短链）时，文案提取偶发失败。
|
||||
- 直接粘贴地址栏链接（如 `jingxuan?modal_id=...`）时，提取成功。
|
||||
|
||||
### 根因
|
||||
|
||||
- `backend/app/modules/tools/service.py` 中 `_download_douyin_manual` 原先只按 `/video/{id}` 提取视频 ID。
|
||||
- 短链重定向结果并不总是 `/video/{id}`，常见还包括：
|
||||
- `/share/video/{id}`
|
||||
- `/user/...?...&vid={id}`
|
||||
- `/follow/search?...&modal_id={id}`
|
||||
- 当落到上述形态时会出现 `Could not extract video_id`，导致 fallback 失败。
|
||||
|
||||
---
|
||||
|
||||
## ✅ 2) 修复方案
|
||||
|
||||
### 2.1 抽取统一解析函数
|
||||
|
||||
- 新增 `_extract_douyin_video_id(candidate_url)`，统一解析以下 ID 形态：
|
||||
- 路径：`/video/{id}`、`/share/video/{id}`
|
||||
- Query 参数：`modal_id`、`vid`、`video_id`、`aweme_id`、`item_id`
|
||||
- 解码后的整串 URL 兜底正则匹配
|
||||
|
||||
### 2.2 fallback 提取链路增强
|
||||
|
||||
- `_download_douyin_manual` 改为：
|
||||
1. 优先从重定向后的 `final_url` 提取 `video_id`
|
||||
2. 若失败，再从原始输入 `url` 提取 `video_id`
|
||||
- 保持后续下载链路不变：访问 `m.douyin.com/share/video/{video_id}` 提取 `play_addr` 并下载。
|
||||
|
||||
---
|
||||
|
||||
## 📁 今日修改文件
|
||||
|
||||
| 文件 | 改动 |
|
||||
|------|------|
|
||||
| `backend/app/modules/tools/service.py` | 新增 `_extract_douyin_video_id`；增强抖音 fallback 的 `video_id` 提取策略（兼容 `share/video`、`modal_id`、`vid` 等） |
|
||||
| `Docs/DevLogs/Day33.md` | 新增 Day33 开发日志，记录问题、根因、修复与验证 |
|
||||
|
||||
---
|
||||
|
||||
## 🔍 验证记录
|
||||
|
||||
- `python -m py_compile backend/app/modules/tools/service.py` ✅
|
||||
- URL 解析冒烟（函数级）：
|
||||
- `jingxuan?modal_id=...` 可提取 ✅
|
||||
- `user?...&vid=...` 可提取 ✅
|
||||
- `follow/search?...&modal_id=...` 可提取 ✅
|
||||
- 下载链路冒烟（服务级）：
|
||||
- 用户提供的短链口令文本可成功下载临时视频 ✅
|
||||
- 历史失败样例 `user?...&vid=...` 可成功走通 fallback 下载 ✅
|
||||
|
||||
---
|
||||
|
||||
## ✅ 3) 文案深度学习：抖音抓取 Playwright 降级增强
|
||||
|
||||
### 3.1 问题复盘
|
||||
|
||||
- 在「文案深度学习」博主分析链路里，抖音用户页有时返回 JS 壳页（含 `byted_acrawler`），静态 HTML 提取拿不到 `desc`。
|
||||
- 表现为：短链可解析 `sec_uid`，但标题抓取报错“页面结构可能已变更”。
|
||||
|
||||
### 3.2 修复方案
|
||||
|
||||
- 在 `backend/app/services/creator_scraper.py` 中新增 Playwright 降级抓取：
|
||||
1. 保留原 HTTP + `ttwid` 抓取作为首选（轻量、快）。
|
||||
2. 当 HTTP 提取不到标题时，自动切换 Playwright。
|
||||
3. 监听页面网络响应，定向捕获：
|
||||
- `/aweme/v1/web/aweme/post/`
|
||||
- `/aweme/v1/web/user/profile/other/`
|
||||
4. 解析响应 JSON 中 `desc` 作为视频标题来源，并提取博主昵称。
|
||||
- 仅在确实失败时返回更准确提示：
|
||||
- `抖音触发风控验证，暂时无法抓取标题，请稍后重试`
|
||||
|
||||
### 3.3 结果
|
||||
|
||||
- 给定短链 `https://v.douyin.com/hmFXdx5PvzQ/` 可稳定识别并完成标题抓取。
|
||||
- 抓取结果可获得有效博主昵称与约 50 条标题（受平台返回数据影响）。
|
||||
|
||||
### 3.4 本次新增/更新文件
|
||||
|
||||
| 文件 | 改动 |
|
||||
|------|------|
|
||||
| `backend/app/services/creator_scraper.py` | 新增抖音 Playwright 降级抓取、网络响应采集、标题/昵称解析优化、错误提示优化 |
|
||||
| `Docs/DevLogs/Day33.md` | 增补文案深度学习抖音抓取增强记录 |
|
||||
|
||||
### 3.5 验证记录
|
||||
|
||||
- `python -m py_compile backend/app/services/creator_scraper.py` ✅
|
||||
- 冒烟验证：
|
||||
- 短链重定向 + `sec_uid` 提取 ✅
|
||||
- HTTP 首选链路失败时自动切换 Playwright ✅
|
||||
- Playwright 网络响应中抓取到 `aweme/post` 数据并提取标题 ✅
|
||||
|
||||
---
|
||||
|
||||
## ✅ 4) 文案深度学习功能首版落地
|
||||
|
||||
### 4.1 后端实现
|
||||
|
||||
- 新增博主抓取服务：`backend/app/services/creator_scraper.py`
|
||||
- `scrape_creator_titles(url)`：平台识别 + 标题抓取统一入口
|
||||
- `validate_url(url)`：`https` 强制、域名白名单、DNS 全记录公网校验、逐跳重定向校验
|
||||
- `cache_titles(titles, user_id)` / `get_cached_titles(analysis_id, user_id)`：20 分钟 TTL + 用户绑定
|
||||
- GLM 服务扩展：`backend/app/services/glm_service.py`
|
||||
- `analyze_topics(titles)`：从标题归纳热门话题（≤10）
|
||||
- `generate_script_from_topic(topic, word_count, titles)`：按话题与风格生成文案
|
||||
- 工具路由新增接口：`backend/app/modules/tools/router.py`
|
||||
- `POST /api/tools/analyze-creator`
|
||||
- `POST /api/tools/generate-topic-script`
|
||||
- 使用 Pydantic JSON 请求模型 + 登录态校验 + 统一 `success_response`
|
||||
|
||||
### 4.2 前端实现
|
||||
|
||||
- 新增状态逻辑 Hook：`frontend/src/features/home/ui/script-learning/useScriptLearning.ts`
|
||||
- 流程状态：`input -> analyzing -> topics -> generating -> result`
|
||||
- 管理分析请求、生成请求、错误态、复制、重新生成
|
||||
- 新增弹窗组件：`frontend/src/features/home/ui/ScriptLearningModal.tsx`
|
||||
- 步骤式 UI：输入链接、话题单选、字数输入、结果展示、填入文案/复制
|
||||
- 接入首页交互：
|
||||
- `frontend/src/features/home/ui/ScriptEditor.tsx`：新增「文案深度学习」按钮
|
||||
- `frontend/src/features/home/model/useHomeController.ts`：新增 `learningModalOpen` 状态
|
||||
- `frontend/src/features/home/ui/HomePage.tsx`：挂载弹窗并支持回填主编辑器
|
||||
|
||||
### 4.3 交互位置与规则
|
||||
|
||||
- 按钮位置已按约定落位：
|
||||
- `历史文案` → `文案提取助手` → `文案深度学习` → `AI多语言`
|
||||
- 弹窗遵循当前统一策略：支持遮罩点击关闭（非关键流程弹窗）。
|
||||
|
||||
### 4.4 验证记录
|
||||
|
||||
- 后端语法检查：
|
||||
- `python -m py_compile backend/app/services/creator_scraper.py backend/app/services/glm_service.py backend/app/modules/tools/router.py` ✅
|
||||
- 前端构建：
|
||||
- `cd frontend && npm run build` ✅
|
||||
- 抖音短链样例联调：
|
||||
- `https://v.douyin.com/hmFXdx5PvzQ/` 可解析、可抓取标题（触发降级时可自动走 Playwright）✅
|
||||
|
||||
---
|
||||
|
||||
## ✅ 5) 抖音 Cookie 依赖澄清与 B站频率限制增强
|
||||
|
||||
### 5.1 抖音 Cookie 依赖澄清
|
||||
|
||||
- 文案深度学习的抖音抓取**不依赖发布管理页登录 Cookie**。
|
||||
- 当前链路使用：
|
||||
- 短链解析 + `sec_uid` 提取
|
||||
- 公共访问链路（`ttwid` + 页面/接口抓取）
|
||||
- 必要时 Playwright 降级
|
||||
- 因此用户即使未登录抖音，也可使用该功能（但仍可能受平台风控影响）。
|
||||
|
||||
### 5.2 B站“请求过于频繁”优化
|
||||
|
||||
- 在 `backend/app/services/creator_scraper.py` 增强 B站抓取稳健性：
|
||||
- 对频率限制场景增加自动重试（指数退避 + 随机抖动）
|
||||
- 频率限制识别（HTTP 412/429、错误码/错误文案）
|
||||
- HTTP 链路失败后自动切换 Playwright 降级抓取
|
||||
- 最终报错文案统一为更可理解的提示
|
||||
- `mid` 提取兼容根路径与子路径（如 `/upload/video`）
|
||||
|
||||
### 5.3 验证记录
|
||||
|
||||
- B站样例联调：`https://space.bilibili.com/8047632` 可抓取 50 条标题 ✅
|
||||
- 抖音短链复测：`https://v.douyin.com/hmFXdx5PvzQ/` 仍可抓取 50 条标题 ✅
|
||||
|
||||
---
|
||||
|
||||
## ✅ 6) 抖音 + B站 抓取可靠性二次增强
|
||||
|
||||
### 6.1 抖音增强
|
||||
|
||||
- `backend/app/services/creator_scraper.py`
|
||||
- `scrape_creator_titles(..., user_id)` 透传用户 ID，支持读取用户已登录平台 Cookie 作为增强上下文。
|
||||
- 抖音抓取新增可选用户 Cookie 注入（HTTP 请求 + Playwright 上下文）。
|
||||
- Playwright 降级抓取轮次从 4 次提升到 8 次，目标改为尽量补齐 `MAX_TITLES=50`。
|
||||
- 保留网络响应抓取主链路（`aweme/post` + `profile/other`），优先 `desc` 提取标题。
|
||||
|
||||
### 6.2 B站增强
|
||||
|
||||
- 新增 WBI 签名链路（主链路）：
|
||||
- 获取 `wbi_img` key（兼容 `nav` 返回 `-101` 但携带 `wbi_img` 的场景）
|
||||
- 计算 `w_rid/wts` 后调用 `x/space/wbi/arc/search`
|
||||
- 多页拉取（分页累加）+ 标题去重，尽量补齐 50 条
|
||||
- 新增 B站会话预热：
|
||||
- `x/frontend/finger/spi` 获取并注入 `buvid3/buvid4`
|
||||
- 支持读取用户已登录 B站 Cookie（若存在）提升命中率
|
||||
- Playwright 降级增强：
|
||||
- 监听 `x/space/*/arc/search` 响应并解析有效 payload
|
||||
- 对捕获的 arc URL 进行 `context.request` 二次回放尝试
|
||||
|
||||
### 6.3 路由联动
|
||||
|
||||
- `backend/app/modules/tools/router.py`
|
||||
- `/api/tools/analyze-creator` 调用抓取时传入 `current_user.id`，用于平台 Cookie 增强。
|
||||
|
||||
### 6.4 结果说明
|
||||
|
||||
- 抖音：短链场景稳定性进一步提升，风控页下优先走 Playwright 降级抓取。
|
||||
- B站：已补齐签名链路与降级链路；但在平台强风控窗口仍可能返回“请求过于频繁/风控校验失败”，属于平台侧限制。
|
||||
|
||||
---
|
||||
|
||||
## ✅ 7) 抓取策略最终调整：抖音/B站改为 Playwright 直连
|
||||
|
||||
根据产品决策，将文案深度学习的博主标题抓取策略统一为 **Playwright 直连主链路**，不再使用“HTTP 主链路 + Playwright 降级”。
|
||||
|
||||
### 7.1 调整内容
|
||||
|
||||
- `backend/app/services/creator_scraper.py`
|
||||
- `_scrape_douyin()` 改为直接调用 `_scrape_douyin_with_playwright()`。
|
||||
- `_scrape_bilibili()` 改为直接调用 `_scrape_bilibili_with_playwright()`。
|
||||
- 两个平台均保留 2 次 Playwright 抓取重试。
|
||||
- 支持优先读取用户隔离 Cookie，若缺失再尝试旧版全局 Cookie。
|
||||
- `backend/app/modules/tools/router.py`
|
||||
- `analyze-creator` 继续传入 `current_user.id`，用于匹配用户 Cookie 上下文。
|
||||
|
||||
### 7.2 影响评估
|
||||
|
||||
- 影响范围仅限「文案深度学习」抓取链路。
|
||||
- **不影响**：视频自动化发布、文案提取助手（extract-script）现有流程。
|
||||
|
||||
### 7.3 验证
|
||||
|
||||
- 抖音短链样例：`https://v.douyin.com/hmFXdx5PvzQ/` 抓取成功，50 条。
|
||||
- B站样例：
|
||||
- `https://space.bilibili.com/256237759?spm_id_from=...` 抓取成功，40 条。
|
||||
- `https://space.bilibili.com/1140672573` 抓取成功，40 条。
|
||||
|
||||
---
|
||||
|
||||
## ✅ 8) GLM 调用链统一与超时体验优化
|
||||
|
||||
### 8.1 现象
|
||||
|
||||
- 文案深度学习“生成文案”偶发前端报错：`timeout of 30000ms exceeded`。
|
||||
|
||||
### 8.2 原因
|
||||
|
||||
- 主要是前端请求超时阈值过短（30s），在模型排队或长文本生成时容易超时。
|
||||
- 后端虽然统一走 `glm_service`，但各方法内部仍重复编写 SDK 调用代码，维护成本高。
|
||||
|
||||
### 8.3 调整
|
||||
|
||||
- 前端：`generate-topic-script` 超时从 30s 提升到 90s，并优化超时提示文案。
|
||||
- 后端：`backend/app/services/glm_service.py`
|
||||
- 新增 `_call_glm(...)` 作为统一调用入口（统一 model / thinking / to_thread / timeout）
|
||||
- `generate_title_tags / rewrite_script / analyze_topics / generate_script_from_topic / translate_text`
|
||||
全部改为复用该入口
|
||||
- 保持 `settings.GLM_MODEL` 单点配置，避免多处散落调用
|
||||
|
||||
### 8.4 结果
|
||||
|
||||
- GLM 调用标准统一，后续参数调整只需改一处。
|
||||
- 前端超时报错显著减少；如确实超时会给出可理解提示。
|
||||
|
||||
---
|
||||
|
||||
## ✅ 9) 三个文案弹窗操作按钮统一
|
||||
|
||||
### 9.1 目标
|
||||
|
||||
- 统一「文案提取助手」「AI 智能改写」「文案深度学习」结果页操作按钮的位置、样式与主次关系。
|
||||
|
||||
### 9.2 调整
|
||||
|
||||
- `frontend/src/features/home/ui/ScriptExtractionModal.tsx`
|
||||
- 结果页按钮从“分散在标题右侧 + 底部单独按钮”改为统一底部 Action Grid。
|
||||
- 按钮统一为：`填入文案`、`复制`、`提取下一个`、`关闭`。
|
||||
- `frontend/src/features/home/ui/RewriteModal.tsx`
|
||||
- 结果页按钮改为统一底部 Action Grid。
|
||||
- 新增复制按钮（含 clipboard fallback）。
|
||||
- 按钮统一为：`填入文案`、`复制`、`重新生成`、`保留原文`。
|
||||
- `frontend/src/features/home/ui/ScriptLearningModal.tsx`
|
||||
- 维持同一 Action Grid 风格：`填入文案`、`复制`、`重新生成`、`换个话题`。
|
||||
|
||||
### 9.3 验证
|
||||
|
||||
- `cd frontend && npm run build` ✅
|
||||
244
Docs/DevLogs/Day34.md
Normal file
244
Docs/DevLogs/Day34.md
Normal file
@@ -0,0 +1,244 @@
|
||||
## 多镜头（Multi-Camera）时间轴系统重构 (Day 34)
|
||||
|
||||
### 概述
|
||||
|
||||
将时间轴系统从"等分顺序片段"模型重构为"主素材 + 插入镜头"多镜头模型。主素材连续循环播放填满整条时间轴，用户可在任意位置叠加插入镜头，实现多机位切换效果。单素材模式行为完全不变。同时补充修复「文案深度学习」弹窗误触关闭问题。
|
||||
|
||||
---
|
||||
|
||||
## ✅ 1) 核心架构变更
|
||||
|
||||
### 1.1 旧模型 vs 新模型
|
||||
|
||||
| | 旧模型 | 新模型 |
|
||||
|---|---|---|
|
||||
| 时间轴结构 | 等分 N 段，每段对应一个素材 | 主素材连续播放 + 浮动插入块 |
|
||||
| 主素材 | 无概念 | `selectedMaterials[0]`，循环填满整条音频时长 |
|
||||
| 其余素材 | 平均分配时长 | 作为插入候选，可自由添加到时间轴任意位置 |
|
||||
| 片段边界 | 固定等分 | 用户拖拽调整位置，点击弹窗编辑时长 |
|
||||
| 最大素材数 | 4（等分） | 4（1 主 + 最多 3 插入候选），每个候选可多次插入 |
|
||||
|
||||
### 1.2 `buildAssignments()` 核心算法
|
||||
|
||||
多素材模式下调用 `toCustomAssignments()` 生成 `custom_assignments` 数组：
|
||||
|
||||
1. 将插入块按 `start` 排序
|
||||
2. 插入块之间的空白（gap）由主素材填充
|
||||
3. 主素材使用 `primaryAccum` 追踪累计播放位置，实现无缝循环
|
||||
4. 每段 gap 按主素材有效片段长度做**边界分割**，确保每个子段不跨越 loop 边界
|
||||
5. 后端 `prepare_segment` 只需做简单裁剪，避免触发"先裁后循环"的帧重复路径
|
||||
|
||||
---
|
||||
|
||||
## ✅ 2) 前端改动
|
||||
|
||||
### 2.1 新增文件
|
||||
|
||||
**`frontend/src/shared/types/timeline.ts`**
|
||||
|
||||
```typescript
|
||||
export interface InsertSegment {
|
||||
id: string;
|
||||
materialId: string;
|
||||
materialName: string;
|
||||
start: number;
|
||||
end: number;
|
||||
sourceStart: number;
|
||||
sourceEnd: number;
|
||||
color: string;
|
||||
}
|
||||
```
|
||||
|
||||
跨模块共享类型，供 `useTimelineEditor`、`TimelineEditor`、`useHomeController` 共用。
|
||||
|
||||
### 2.2 `useTimelineEditor.ts` — 完全重写
|
||||
|
||||
核心 Hook 从等分模型重写为主素材+插入模型：
|
||||
|
||||
- **新 API**：`addInsert`（返回 `AddInsertResult: "ok" | "limit" | "no_space"`）、`removeInsert`、`moveInsert`、`resizeInsert`、`setInsertSourceRange`、`setPrimarySourceRange`、`toCustomAssignments`
|
||||
- **`MultiCamCache`** 接口：独立 localStorage 持久化（`vigent_${storageKey}_multicam`），保存 inserts + primarySourceStart/End
|
||||
- 自动清理：当选中素材列表变化时，移除引用已删除素材的插入块
|
||||
- 主素材源范围在单/多模式切换时自动重置
|
||||
|
||||
### 2.3 `TimelineEditor.tsx` — 完全重写
|
||||
|
||||
可视化组件配合新模型：
|
||||
|
||||
- 主素材背景条：紫色底色 + 循环条纹图案（loopCount > 1 时显示）
|
||||
- 浮动插入块：彩色半透明矩形，支持拖拽移动（中央），点击弹出 ClipTrimmer 编辑截取范围与时长
|
||||
- 插入候选栏：`selectedMaterials[1:]` 显示为 `+` 按钮，点击添加到时间轴
|
||||
- 移动端适配：40px 最小高度、12px 拖拽边缘、始终可见的删除按钮
|
||||
- 清理了未使用的 `TimelineSegment` import
|
||||
|
||||
### 2.4 `useHomeController.ts` — 适配新 API
|
||||
|
||||
- 替换旧 timeline 解构为新 API（`inserts`、`addInsert`、`removeInsert` 等）
|
||||
- `handleGenerate()` 多素材分支重写：调用 `toCustomAssignments()` 生成 assignments，构建 payload 时拆分 `material_path`（主）和 `material_paths`（全部去重路径）
|
||||
- 单素材分支同样调用 `toCustomAssignments()` 处理裁剪范围
|
||||
- 素材重命名时同步更新 inserts 中的 `materialName`
|
||||
- 新增 `handleSetPrimary` 回调：将指定素材提升到 `selectedMaterials[0]`
|
||||
- 新增 `insertCandidates` 计算值：`selectedMaterials[1:]` 对应的 Material 对象列表
|
||||
|
||||
### 2.5 `MaterialSelector.tsx` — 增强
|
||||
|
||||
- 新增 `Crown` 图标和 `onSetPrimary` 回调 prop
|
||||
- 多素材模式下显示角色标签：`selectedMaterials[0]` 显示紫色"主素材"徽章，其余显示灰色"可插入"徽章
|
||||
- 非主素材行显示 Crown 按钮，点击可设为主素材
|
||||
|
||||
### 2.6 `HomePage.tsx` — 适配
|
||||
|
||||
- `clipTrimmerSegment` 重写：支持 `"primary"` ID（主素材裁剪）和插入块 ID 两种路由
|
||||
- `TimelineEditor` 组件传入全部新 props
|
||||
- `ClipTrimmer` 的 `onConfirm` 根据 segment ID 路由到 `setPrimarySourceRange` 或 `setInsertSourceRange`
|
||||
- `MaterialSelector` 传入 `onSetPrimary`
|
||||
|
||||
---
|
||||
|
||||
## ✅ 3) 后端改动
|
||||
|
||||
### 3.1 `workflow.py` — 多镜头支持修复
|
||||
|
||||
四项关键修复：
|
||||
|
||||
**(a) material_paths 来源**
|
||||
|
||||
```python
|
||||
# 旧：从 custom_assignments 推断（不适用于多镜头）
|
||||
# 新：优先信任前端传入的 req.material_paths
|
||||
if req.material_paths and len(req.material_paths) >= 1:
|
||||
material_paths = req.material_paths
|
||||
else:
|
||||
material_paths = [req.material_path]
|
||||
```
|
||||
|
||||
**(b) custom_assignments 校验**
|
||||
|
||||
```python
|
||||
# 旧：len(custom_assignments) == len(material_paths)
|
||||
# 新：>= 1 + 硬上限 50 + 路径子集校验
|
||||
if len(req.custom_assignments) > 50:
|
||||
raise ValueError(...)
|
||||
unknown = [a.material_path for a in req.custom_assignments if a.material_path not in known_paths]
|
||||
if unknown:
|
||||
raise ValueError(...)
|
||||
```
|
||||
|
||||
**(c) 下载去重 + 并发控制**
|
||||
|
||||
```python
|
||||
# 旧：每个 assignment 独立下载（同一素材重复下载）
|
||||
# 新：按唯一路径去重下载，path_to_local 映射
|
||||
_segment_sem = asyncio.Semaphore(4) # 每次调用内部创建，非模块级
|
||||
unique_paths = list(dict.fromkeys(a["material_path"] for a in assignments))
|
||||
path_to_local: dict = {}
|
||||
```
|
||||
|
||||
Semaphore 在每次 `generate_video()` 内部创建，2 个并发任务 × 4 = 峰值 8 个 ffmpeg 进程。
|
||||
|
||||
**(d) 首尾段 capping 保护**
|
||||
|
||||
```python
|
||||
# 仅在非 custom_assignments 模式下执行首尾对齐
|
||||
if not req.custom_assignments and assignments and audio_duration > 0:
|
||||
assignments[0]["start"] = 0.0
|
||||
assignments[-1]["end"] = audio_duration
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ 4) 文案深度学习弹窗防误触关闭
|
||||
|
||||
### 4.1 问题
|
||||
|
||||
- 「文案深度学习」弹窗默认支持遮罩与 `ESC` 关闭，用户在查看生成结果时容易误触关闭，重新打开后已生成内容丢失。
|
||||
|
||||
### 4.2 修复
|
||||
|
||||
- `frontend/src/shared/ui/AppModal.tsx`
|
||||
- 新增 `closeOnEsc?: boolean` 配置，默认值 `true`，保持旧弹窗行为不变。
|
||||
- `frontend/src/features/home/ui/ScriptLearningModal.tsx`
|
||||
- 设置 `closeOnOverlay={false}` 与 `closeOnEsc={false}`，禁止遮罩/ESC 关闭。
|
||||
- 输入页底部按钮由“取消”改为“清空”，仅清理链接输入，不关闭弹窗。
|
||||
- 关闭路径收敛为：右上角 `X` 或结果页“填入文案”。
|
||||
|
||||
---
|
||||
|
||||
## ✅ 5) Code Review 修复
|
||||
|
||||
### 5.1 UX：统一时长编辑入口
|
||||
|
||||
- **问题**：时间轴插入块同时支持右边缘拖拽调时长和点击弹窗编辑，拖拽操作每次都误触弹窗
|
||||
- **修复**：
|
||||
- 移除 `TimelineEditor` 右侧 resize handle
|
||||
- 引入 `dragMovedRef` + 5px 像素阈值区分拖拽与点击
|
||||
- `ClipTrimmer` onConfirm 新增 `resizeInsert()` 同步，确认截取后自动更新时间轴块时长
|
||||
- 帮助文字更新："点击插入块设置截取/时长"
|
||||
|
||||
### 5.2 Lint 修复
|
||||
|
||||
- `useTimelineEditor.ts`：3 处 `react-hooks/set-state-in-effect`，用 `eslint-disable-next-line` 标注（初始化和清理场景）
|
||||
- `useTimelineEditor.ts`：render-time ref 访问改为 `useState` 模式（`prevPrimaryId`）
|
||||
- `HomePage.tsx`：移除未使用的 `reorderMaterials` 解构
|
||||
- `TimelineEditor.tsx`：移除未使用的 `useMemo` import 和 `materials`/`onResizeInsert` props
|
||||
|
||||
### 5.3 P1：多片段 assignment 退化
|
||||
|
||||
- **问题**：`selectedMaterials.length > 1` 但时间轴无插入块时，`is_multi=False`，后端走单素材路径丢弃非主素材
|
||||
- **修复**（`workflow.py`）：
|
||||
|
||||
```python
|
||||
is_multi = len(material_paths) > 1 or (
|
||||
req.custom_assignments is not None and len(req.custom_assignments) > 1
|
||||
)
|
||||
```
|
||||
|
||||
### 5.4 P1：主素材 trim range 泄漏
|
||||
|
||||
- **问题**：切换主素材（"设为主素材"）时，旧主素材的 `primarySourceStart/End` 保留给新主素材，导致截取范围错误
|
||||
- **原因**：仅按 `selectedMaterials.length` 变化重置，切换主素材时长度不变
|
||||
- **修复**（`useTimelineEditor.ts`）：改用 identity 追踪
|
||||
|
||||
```typescript
|
||||
const [prevPrimaryId, setPrevPrimaryId] = useState(selectedMaterials[0]);
|
||||
if (selectedMaterials[0] !== prevPrimaryId) {
|
||||
setPrevPrimaryId(selectedMaterials[0]);
|
||||
setPrimarySourceStart(0);
|
||||
setPrimarySourceEnd(0);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📁 今日修改文件
|
||||
|
||||
| 文件 | 改动 |
|
||||
|------|------|
|
||||
| `frontend/src/shared/types/timeline.ts` | **新增**：`InsertSegment` 接口定义 |
|
||||
| `frontend/src/features/home/model/useTimelineEditor.ts` | **重写**：等分模型 → 主素材+插入模型 |
|
||||
| `frontend/src/features/home/ui/TimelineEditor.tsx` | **重写**：可视化组件适配新模型 |
|
||||
| `frontend/src/features/home/model/useHomeController.ts` | 适配新 timeline API、生成 payload 重写 |
|
||||
| `frontend/src/features/home/ui/MaterialSelector.tsx` | 主素材/可插入标签、设为主素材按钮 |
|
||||
| `frontend/src/features/home/ui/HomePage.tsx` | ClipTrimmer 路由、TimelineEditor 新 props |
|
||||
| `backend/app/modules/videos/workflow.py` | material_paths 来源、校验、下载去重、capping 保护 |
|
||||
| `frontend/src/shared/ui/AppModal.tsx` | 新增 `closeOnEsc` 配置，支持按弹窗粒度控制 ESC 关闭行为 |
|
||||
| `frontend/src/features/home/ui/ScriptLearningModal.tsx` | 禁用遮罩/ESC 关闭；输入页“取消”改为“清空” |
|
||||
|
||||
---
|
||||
|
||||
## 🔍 验证记录
|
||||
|
||||
- TypeScript 编译检查：`npx tsc --noEmit` ✅ 无错误
|
||||
- Python 语法检查：`python -c "import ast; ast.parse(open(...).read())"` ✅
|
||||
- 前端 lint（本次补充修复）：`npm run lint -- src/shared/ui/AppModal.tsx src/features/home/ui/ScriptLearningModal.tsx` ✅
|
||||
- 代码审查（前端 + 后端各一轮 subagent review）：
|
||||
- 前端：逻辑正确，无 bug，仅 1 处未使用 import（已清理）
|
||||
- 后端：校验逻辑、下载去重、并发控制均正确
|
||||
- 单素材模式向后兼容：`toCustomAssignments()` 在单素材时正确生成带裁剪范围的单段 assignment
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ 已知限制
|
||||
|
||||
- `prepare_segment` 的"先裁后循环"路径（`needs_loop && source_start > 0`）仍存在，但前端的边界分割算法确保永远不会触发该路径
|
||||
- 插入块最多 10 个（`useTimelineEditor` 内 `MAX_INSERTS=10`），超出时返回 `"limit"`
|
||||
- 插入块最小时长 0.5s，低于此值的操作会被忽略
|
||||
@@ -39,8 +39,12 @@ frontend/src/
|
||||
│ │ ├── MaterialSelector.tsx
|
||||
│ │ ├── ScriptEditor.tsx
|
||||
│ │ ├── ScriptExtractionModal.tsx
|
||||
│ │ ├── RewriteModal.tsx
|
||||
│ │ ├── ScriptLearningModal.tsx
|
||||
│ │ ├── script-extraction/
|
||||
│ │ │ └── useScriptExtraction.ts
|
||||
│ │ ├── script-learning/
|
||||
│ │ │ └── useScriptLearning.ts
|
||||
│ │ ├── TitleSubtitlePanel.tsx
|
||||
│ │ ├── FloatingStylePreview.tsx
|
||||
│ │ ├── VoiceSelector.tsx
|
||||
@@ -69,7 +73,8 @@ frontend/src/
|
||||
│ │ ├── useTitleInput.ts
|
||||
│ │ └── usePublishPrefetch.ts
|
||||
│ ├── ui/
|
||||
│ │ └── SelectPopover.tsx # 统一下拉/BottomSheet 选择器
|
||||
│ │ ├── SelectPopover.tsx # 统一下拉/BottomSheet 选择器
|
||||
│ │ └── AppModal.tsx # 统一弹窗基座
|
||||
│ ├── types/
|
||||
│ │ ├── user.ts # User 类型定义
|
||||
│ │ └── publish.ts # 发布相关类型
|
||||
@@ -213,7 +218,7 @@ body {
|
||||
|
||||
## 统一弹窗规范 (AppModal)
|
||||
|
||||
所有居中弹窗（如视频预览、文案提取、AI 改写、文案扩展编辑、录音、密码修改）统一使用 `@/shared/ui/AppModal` + `AppModalHeader`：
|
||||
所有居中弹窗（如视频预览、文案提取、AI 改写、文案深度学习、文案扩展编辑、录音、密码修改）统一使用 `@/shared/ui/AppModal` + `AppModalHeader`：
|
||||
|
||||
- 统一遮罩与层级：`fixed inset-0` + `bg-black/80` + `backdrop-blur-sm` + 明确 `z-index`
|
||||
- 统一挂载位置：通过 Portal 挂载到 `document.body`，避免局部容器/层叠上下文影响，确保是全页面弹窗
|
||||
@@ -225,6 +230,19 @@ body {
|
||||
- 统一滚动策略：弹窗打开时锁定背景滚动（`lockBodyScroll`），内容区自行滚动
|
||||
- 特殊层级场景（例如视频预览压过下拉）使用更高 `z-index`（如 `z-[320]`）
|
||||
|
||||
### 文案类弹窗结果操作栏规范
|
||||
|
||||
适用组件：
|
||||
- `ScriptExtractionModal`
|
||||
- `RewriteModal`
|
||||
- `ScriptLearningModal`
|
||||
|
||||
统一要求：
|
||||
- 结果页操作按钮统一放在内容底部（Action Grid），避免“标题右上角按钮 + 底部按钮”混排。
|
||||
- 主按钮统一为高亮渐变（如「填入文案」），其余按钮统一次级样式（`bg-white/10`）。
|
||||
- 动作文案尽量统一：`填入文案` / `复制` / `重新生成`（或与当前流程等价的返回动作）。
|
||||
- 按钮尺寸、圆角、间距保持一致（推荐 `py-2.5 px-3 rounded-lg text-sm`）。
|
||||
|
||||
---
|
||||
|
||||
## 发布后清理弹窗规范 (CleanupContext)
|
||||
@@ -248,7 +266,7 @@ body {
|
||||
所有需要认证的 API 请求**必须**使用 `@/shared/api/axios` 导出的 axios 实例。该实例已配置：
|
||||
- 自动携带 `credentials: include`
|
||||
- 遇到 401/403 时自动清除 cookie 并跳转登录页
|
||||
- AI/Tools 接口（如 `/api/ai/*`、`/api/tools/extract-script`）现为强制鉴权，禁止匿名 `fetch` 直调
|
||||
- AI/Tools 接口（如 `/api/ai/*`、`/api/tools/extract-script`、`/api/tools/analyze-creator`、`/api/tools/generate-topic-script`）现为强制鉴权，禁止匿名 `fetch` 直调
|
||||
|
||||
**使用方式：**
|
||||
|
||||
|
||||
@@ -57,9 +57,9 @@ ViGent2 的前端界面，采用 Next.js 16 + TailwindCSS 构建。
|
||||
### 4. 配音前置 + 时间轴编排
|
||||
- **配音独立生成**: 先生成配音 → 选中配音 → 再选素材 → 生成视频。
|
||||
- **配音管理面板**: 生成/试听/改名/删除/选中，异步生成 + 进度轮询。
|
||||
- **时间轴编辑器**: wavesurfer.js 音频波形 + 色块可视化素材分配，拖拽分割线调整各段时长。
|
||||
- **素材截取设置**: ClipTrimmer 双手柄 range slider + HTML5 视频预览播放。
|
||||
- **拖拽排序**: 时间轴色块支持 HTML5 Drag & Drop 调换素材顺序。
|
||||
- **时间轴编辑器**: wavesurfer.js 音频波形 + 主素材连续播放背景 + 浮动插入镜头块，拖拽移动位置，点击弹窗编辑截取范围与时长。
|
||||
- **素材截取设置**: ClipTrimmer 双手柄 range slider + HTML5 视频预览播放（主素材与插入块统一入口）。
|
||||
- **多镜头模型**: 主素材循环填满音频时长，其余素材作为插入候选可多次添加到时间轴任意位置；支持"设为主素材"切换。
|
||||
- **自定义分配**: 后端 `custom_assignments` 支持用户定义的素材分配方案（含 `source_start/source_end` 截取区间）。
|
||||
- **时间轴语义对齐**: 超出音频时仅保留可见段并截齐末段，超出段不参与生成；不足音频时最后可见段自动循环补齐。
|
||||
- **画面比例控制**: 时间轴顶部支持 `9:16 / 16:9` 输出比例选择，设置持久化并透传后端。
|
||||
@@ -91,13 +91,12 @@ ViGent2 的前端界面，采用 Next.js 16 + TailwindCSS 构建。
|
||||
- **到期续费**: 会员到期后登录自动跳转付费页续费，流程与首次开通一致。
|
||||
- **管理员激活**: 管理员手动激活功能并存，两种方式互不影响。
|
||||
|
||||
### 9. 文案提取助手 (`ScriptExtractionModal`)
|
||||
- **多源提取**: 支持文件拖拽上传与 URL 粘贴 (B站/抖音/TikTok)。
|
||||
- **AI 智能改写**: 集成 GLM-4.7-Flash，自动改写为口播文案。
|
||||
- **登录鉴权**: 依赖后端受保护接口（`/api/tools/extract-script`），未登录会触发全局 401 跳转登录。
|
||||
- **自定义提示词**: 可自定义改写提示词，留空使用默认；设置持久化到 localStorage。
|
||||
- **一键填入**: 提取结果直接填充至视频生成输入框。
|
||||
- **智能交互**: 实时进度展示，防误触设计。
|
||||
### 9. 文案创作助手（3 个弹窗）
|
||||
- **文案提取助手** (`ScriptExtractionModal`): 支持文件上传与 URL 提取（需登录），提取结果可一键填入主编辑器。
|
||||
- **AI 智能改写** (`RewriteModal`): 基于 GLM-4.7-Flash 改写文案，支持自定义提示词持久化。
|
||||
- **文案深度学习** (`ScriptLearningModal`): 输入抖音/B站博主主页，分析热门话题并生成口播文案（需登录）。
|
||||
- **统一结果操作栏**: 三个弹窗结果页统一底部 Action Grid 风格，主按钮为「填入文案」，次按钮统一「复制 / 重新生成（或等价返回操作）」。
|
||||
- **登录鉴权**: 依赖受保护接口（`/api/tools/*`、`/api/ai/*`），未登录会触发全局 401 跳转登录。
|
||||
|
||||
## 🛠️ 技术栈
|
||||
|
||||
@@ -161,6 +160,7 @@ src/
|
||||
- 业务选择器统一使用 `SelectPopover`（桌面 Popover / 移动端 BottomSheet）；`ScriptEditor` 的“历史文案 / AI多语言”保留原轻量菜单。
|
||||
- 业务弹窗统一使用 `AppModal`（统一遮罩、头部、关闭行为与滚动策略）。
|
||||
- 弹窗关闭策略：默认支持 `ESC` / `X` / 点击空白关闭；仅发布成功清理弹窗为强制流程（不允许空白关闭，也不显示 `X`）。
|
||||
- 文案类弹窗结果页按钮统一：底部 Action Grid、主次按钮层级一致、文案动作命名一致（填入/复制/重新生成）。
|
||||
- 视频预览弹窗层级高于下拉菜单；下拉内支持连续预览。
|
||||
- 页面同时适配桌面端与移动端；长列表统一隐藏滚动条。
|
||||
- 详细 UI 规范、持久化规范与交互约束请查看 `Docs/FRONTEND_DEV.md`。
|
||||
|
||||
@@ -1,16 +1,39 @@
|
||||
# ViGent2 开发任务清单 (Task Log)
|
||||
|
||||
**项目**: ViGent2 数字人口播视频生成系统
|
||||
**进度**: 100% (Day 32 - 视频下载同源修复 + 安全整改第一批)
|
||||
**更新时间**: 2026-03-04
|
||||
|
||||
---
|
||||
|
||||
## 📅 对话历史与开发日志
|
||||
|
||||
> 这里记录了每一天的核心开发内容与 milestone。
|
||||
|
||||
### Day 32: 视频下载同源修复 + 安全整改第一批 + Day 日志拆分归档 (Current)
|
||||
# ViGent2 开发任务清单 (Task Log)
|
||||
|
||||
**项目**: ViGent2 数字人口播视频生成系统
|
||||
**进度**: 100% (Day 34 - 多镜头时间轴重构 + 文案深度学习弹窗防误触关闭)
|
||||
**更新时间**: 2026-03-09
|
||||
|
||||
---
|
||||
|
||||
## 📅 对话历史与开发日志
|
||||
|
||||
> 这里记录了每一天的核心开发内容与 milestone。
|
||||
|
||||
### Day 34: 多镜头时间轴重构 + 文案深度学习弹窗防误触关闭 + Code Review 修复 (Current)
|
||||
- [x] **时间轴模型重构**: 多素材从”等分顺序片段”升级为”主素材连续播放 + 插入镜头块”，支持自由插入、拖拽移动。
|
||||
- [x] **前端链路落地**: 重写 `useTimelineEditor` 与 `TimelineEditor`，新增主素材/插入候选语义，`useHomeController` / `HomePage` / `MaterialSelector` 全链路适配。
|
||||
- [x] **后端生成链路适配**: `workflow.py` 完成 `material_paths` 来源修正、`custom_assignments` 新校验、素材下载去重与段处理并发限制，保持单素材兼容。
|
||||
- [x] **文案深度学习防误触关闭**: `ScriptLearningModal` 禁用遮罩和 `ESC` 关闭，仅允许右上角 `X` 或”填入文案”关闭；输入页”取消”改为”清空”。
|
||||
- [x] **Code Review 修复**:
|
||||
- UX: 移除时间轴 resize handle，统一用 ClipTrimmer 弹窗编辑时长；引入拖拽/点击像素阈值区分。
|
||||
- Lint: 修复 `useTimelineEditor` 3 处 set-state-in-effect、`HomePage` 未使用解构、`TimelineEditor` 未使用 import/props。
|
||||
- P1: `workflow.py` `is_multi` 补充 `custom_assignments` 条件，防止多片段 assignment 退化为单素材路径。
|
||||
- P1: 主素材 trim range 改为按 identity（非 count）重置，修复切换主素材时截取范围泄漏。
|
||||
- ClipTrimmer onConfirm 同步调用 `resizeInsert()` 更新时间轴块时长。
|
||||
- [x] **文档同步**: 回写 `Day34` 与 `TASK_COMPLETE`，并更新 Current 指向。
|
||||
|
||||
### Day 33: 文案深度学习落地 + 抓取稳定性增强 + 交互统一
|
||||
- [x] **文案深度学习功能上线**: 新增 `ScriptLearningModal`（输入主页链接 -> 话题分析 -> 生成文案 -> 填入编辑器）与首页入口接入。
|
||||
- [x] **Tools 新接口**: 新增 `POST /api/tools/analyze-creator` 与 `POST /api/tools/generate-topic-script`，并接入登录鉴权。
|
||||
- [x] **抖音/B站抓取增强**: 博主标题抓取统一升级为 Playwright 直连主链路，支持用户 Cookie 上下文增强与失败重试。
|
||||
- [x] **GLM 调用统一收口**: `glm_service` 新增统一调用入口，标题生成/改写/翻译/话题分析/话题文案生成全部复用，减少重复代码。
|
||||
- [x] **超时体验优化**: 文案深度学习“生成文案”前端超时从 30s 提升到 90s，并补充超时提示文案。
|
||||
- [x] **文案弹窗交互统一**: 文案提取/AI 改写/文案深度学习结果页按钮统一为底部 Action Grid，主次按钮层级与文案动作统一。
|
||||
- [x] **依赖升级**: 后端 venv 升级 `yt-dlp`、`playwright`、`biliup` 并完成兼容性冒烟验证。
|
||||
- [x] **文档同步**: 回写 `Day33`、`FRONTEND_README`、`FRONTEND_DEV`、`BACKEND_README`、`BACKEND_DEV`、`TASK_COMPLETE`。
|
||||
|
||||
### Day 32: 视频下载同源修复 + 安全整改第一批 + Day 日志拆分归档
|
||||
- [x] **下载链路修复**: 新增 `GET /api/videos/generated/{video_id}/download`，统一返回 `Content-Disposition: attachment`，修复“点击下载却新开标签页播放”问题。
|
||||
- [x] **发布成功弹窗下载改造**: `CleanupContext` 从传 URL 改为传 `videoId`，下载按钮改走同源接口，去掉 `target="_blank"`。
|
||||
- [x] **首页下载改造**: `PreviewPanel` 同步切换到同源下载接口，首页与发布页下载行为一致。
|
||||
@@ -42,21 +65,21 @@
|
||||
- [x] **发布后清理链路加固**: 新增/优化 `CleanupContext` + `/api/videos/cleanup` 全链路；后端删除异常不再吞错、清理接口严格成功语义；前端失败不清本地/不关弹窗，3 次失败可暂不清理，清理状态 24h 过期并支持用户切换复位；清理范围收敛为输入内容字段并保留用户偏好。
|
||||
|
||||
### Day 30: Remotion 缓存修复 + 编码流水线质量优化 + 唇形同步容错 + 统一下拉交互
|
||||
- [x] **Remotion 缓存 404 修复**: bundle 缓存命中时，新生成的视频/字体文件不在旧缓存 `public/` 目录 → 404 → 回退 FFmpeg（无标题字幕）。改为硬链接（`fs.linkSync`）当前渲染所需文件到缓存目录。
|
||||
- [x] **LatentSync `read_video` 跳过冗余 FPS 重编码**: 检测输入 FPS，已是 25fps 时跳过 `ffmpeg -r 25 -crf 18` 重编码。
|
||||
- [x] **LatentSync final mux 流复制**: `imageio` CRF 13 写帧后的 mux 步骤从 `libx264 -crf 18` 改为 `-c:v copy`，消除冗余双重编码。
|
||||
- [x] **`prepare_segment` + `normalize_orientation` CRF 提质**: CRF 23 → 18，与 LatentSync 内部质量标准统一。
|
||||
- [x] **多素材 concat 流复制**: 各段参数已统一，`concat_videos` 从 `libx264 -crf 23` 改为 `-c:v copy`。
|
||||
- [x] **编码次数总计**: 从 5-6 次有损编码降至 3 次（prepare_segment → LatentSync/MuseTalk 模型输出 → Remotion）。
|
||||
- [x] **LatentSync 无脸帧容错**: 素材部分帧检测不到人脸时不再中断推理，无脸帧保留原画面，单素材异常时回退原视频。
|
||||
- [x] **MuseTalk 管道直编码**: `cv2.VideoWriter(mp4v)` 中间有损文件改为 FFmpeg rawvideo stdin 管道，消除一次冗余有损编码。
|
||||
- [x] **MuseTalk 参数环境变量化**: 推理与编码参数（detect_every/blend_cache/CRF/preset 等）从硬编码迁移到 `backend/.env`，当前使用质量优先档（CRF 14, preset slow, detect_every 2, blend_cache_every 2）。
|
||||
- [x] **Workflow 异步防阻塞**: 新增 `_run_blocking()` 线程池辅助，5 处同步 FFmpeg 调用（旋转归一化/prepare_segment/concat/BGM 混音）改为 `await _run_blocking()`，事件循环不再被阻塞。
|
||||
- [x] **compose 跳过优化**: 无 BGM 时 `final_audio_path == audio_path`，跳过多余的 compose 步骤，Remotion 路径直接用 lipsync 输出，非 Remotion 路径 `shutil.copy` 透传。
|
||||
- [x] **compose() 异步化**: `compose()` 改为 `async def`，内部 `_get_duration` 和 `_run_ffmpeg` 走 `run_in_executor`。
|
||||
- [x] **同分辨率跳过 scale**: 多素材逐段比对分辨率，匹配的传 `None` 走 copy 分支；单素材同理。避免已是目标分辨率时的无效重编码。
|
||||
- [x] **`_get_duration()` 线程池化**: workflow 中 3 处同步 ffprobe 探测改为 `await _run_blocking()`。
|
||||
- [x] **compose 循环 CRF 统一**: 循环场景 CRF 23 → 18，与全流水线质量标准一致。
|
||||
- [x] **Remotion 缓存 404 修复**: bundle 缓存命中时，新生成的视频/字体文件不在旧缓存 `public/` 目录 → 404 → 回退 FFmpeg（无标题字幕）。改为硬链接（`fs.linkSync`）当前渲染所需文件到缓存目录。
|
||||
- [x] **LatentSync `read_video` 跳过冗余 FPS 重编码**: 检测输入 FPS，已是 25fps 时跳过 `ffmpeg -r 25 -crf 18` 重编码。
|
||||
- [x] **LatentSync final mux 流复制**: `imageio` CRF 13 写帧后的 mux 步骤从 `libx264 -crf 18` 改为 `-c:v copy`，消除冗余双重编码。
|
||||
- [x] **`prepare_segment` + `normalize_orientation` CRF 提质**: CRF 23 → 18，与 LatentSync 内部质量标准统一。
|
||||
- [x] **多素材 concat 流复制**: 各段参数已统一，`concat_videos` 从 `libx264 -crf 23` 改为 `-c:v copy`。
|
||||
- [x] **编码次数总计**: 从 5-6 次有损编码降至 3 次（prepare_segment → LatentSync/MuseTalk 模型输出 → Remotion）。
|
||||
- [x] **LatentSync 无脸帧容错**: 素材部分帧检测不到人脸时不再中断推理，无脸帧保留原画面，单素材异常时回退原视频。
|
||||
- [x] **MuseTalk 管道直编码**: `cv2.VideoWriter(mp4v)` 中间有损文件改为 FFmpeg rawvideo stdin 管道，消除一次冗余有损编码。
|
||||
- [x] **MuseTalk 参数环境变量化**: 推理与编码参数（detect_every/blend_cache/CRF/preset 等）从硬编码迁移到 `backend/.env`，当前使用质量优先档（CRF 14, preset slow, detect_every 2, blend_cache_every 2）。
|
||||
- [x] **Workflow 异步防阻塞**: 新增 `_run_blocking()` 线程池辅助，5 处同步 FFmpeg 调用（旋转归一化/prepare_segment/concat/BGM 混音）改为 `await _run_blocking()`，事件循环不再被阻塞。
|
||||
- [x] **compose 跳过优化**: 无 BGM 时 `final_audio_path == audio_path`，跳过多余的 compose 步骤，Remotion 路径直接用 lipsync 输出，非 Remotion 路径 `shutil.copy` 透传。
|
||||
- [x] **compose() 异步化**: `compose()` 改为 `async def`，内部 `_get_duration` 和 `_run_ffmpeg` 走 `run_in_executor`。
|
||||
- [x] **同分辨率跳过 scale**: 多素材逐段比对分辨率，匹配的传 `None` 走 copy 分支；单素材同理。避免已是目标分辨率时的无效重编码。
|
||||
- [x] **`_get_duration()` 线程池化**: workflow 中 3 处同步 ffprobe 探测改为 `await _run_blocking()`。
|
||||
- [x] **compose 循环 CRF 统一**: 循环场景 CRF 23 → 18，与全流水线质量标准一致。
|
||||
- [x] **多素材片段校验**: prepare 完成后校验片段数量一致，防止空片段进入 concat。
|
||||
- [x] **唇形模型前端选择**: 生成按钮右侧新增模型下拉（默认模型/快速模型/高级模型），全链路透传 `lipsync_model` 到后端路由。默认保持阈值策略，快速强制 MuseTalk，高级强制 LatentSync，三种模式均有 LatentSync 兜底。选择 localStorage 持久化。
|
||||
- [x] **业务下拉统一组件化**: 新增 `SelectPopover`（桌面 Popover + 移动端 BottomSheet），覆盖首页/发布页主要业务选择器（音色、参考音频、配音、素材、BGM、作品、样式、模型、画面比例）。
|
||||
@@ -65,275 +88,275 @@
|
||||
- [x] **BGM 交互收敛**: BGM 选择改为发布页同款（搜索 + 列表 + 试听）；按产品要求移除首页音量滑杆，生成请求固定 `bgm_volume=0.2`。
|
||||
- [x] **例外回退**: `ScriptEditor` 的“历史文案 / AI多语言”恢复原轻量菜单样式（不强制统一 SelectPopover）。
|
||||
- [x] **文档同步**: Day30 / TASK_COMPLETE / FRONTEND_DEV / FRONTEND_README / README / BACKEND_README 同步更新到最终实现。
|
||||
|
||||
### Day 29: 视频流水线优化 + CosyVoice 语气控制
|
||||
- [x] **字幕同步修复**: Whisper 时间戳三步平滑（单调递增+重叠消除+间隙填补）+ 原文节奏映射（线性插值 + 单字时长钳位）。
|
||||
- [x] **LatentSync 嘴型参数调优**: inference_steps 16→20, guidance_scale 2.0, DeepCache 启用, Remotion concurrency 16→4。
|
||||
- [x] **compose 流复制**: 不循环时 `-c:v copy` 替代 libx264 重编码，compose 耗时从分钟级降到秒级。
|
||||
- [x] **FFmpeg 超时保护**: `_run_ffmpeg()` timeout=600, `_get_duration()` timeout=30。
|
||||
- [x] **全局并发限制**: `asyncio.Semaphore(2)` 控制同时运行的生成任务数。
|
||||
- [x] **Redis 任务 TTL**: create 24h, completed/failed 2h, list 自动清理过期索引。
|
||||
- [x] **临时字体清理**: 字体文件加入 temp_files 清理列表。
|
||||
- [x] **预览背景 CORS 修复**: 素材同源代理 `/api/materials/stream/{id}` 彻底绕开跨域。
|
||||
- [x] **CosyVoice 语气控制**: 声音克隆模式新增语气下拉（正常/欢快/低沉/严肃），基于 `inference_instruct2()` 自然语言指令控制情绪，全链路透传 instruct_text，默认"正常"行为不变。
|
||||
|
||||
### Day 28: CosyVoice FP16 加速 + 文档全面更新
|
||||
- [x] **CosyVoice FP16 半精度加速**: `AutoModel()` 开启 `fp16=True`，LLM 推理和 Flow Matching 自动混合精度运行，预估提速 30-40%、显存降低 ~30%。
|
||||
- [x] **文档全面更新**: README.md / DEPLOY_MANUAL.md / SUBTITLE_DEPLOY.md / BACKEND_README.md 补充 MuseTalk 混合唇形同步方案、性能优化、Remotion 并发渲染等内容。
|
||||
|
||||
### Day 27: Remotion 描边修复 + 字体样式扩展 + 混合唇形同步 + 性能优化
|
||||
- [x] **描边渲染修复**: 标题/副标题/字幕从 `textShadow` 4 方向模拟改为 CSS 原生 `-webkit-text-stroke` + `paint-order: stroke fill`，修复描边过粗和副标题重影问题。
|
||||
- [x] **字体样式扩展**: 标题样式 4→12 个（+庞门正道/优设标题圆/阿里数黑体/文道潮黑/无界黑/厚底黑/寒蝉半圆体/欣意吉祥宋），字幕样式 4→8 个（+少女粉/清新绿/金色隶书/楷体红字）。
|
||||
- [x] **描边参数优化**: 所有预设 `stroke_size` 从 8 降至 4~5，配合原生描边视觉更干净。
|
||||
- [x] **TypeScript 类型修复**: Root.tsx `Composition` 泛型与 `calculateMetadata` 参数类型对齐；Video.tsx `VideoProps` 添加索引签名兼容 `Record<string, unknown>`；VideoLayer.tsx 移除 `OffthreadVideo` 不支持的 `loop` prop。
|
||||
- [x] **进度条文案还原**: 进度条从显示后端推送消息改回固定 `正在AI生成中...`。
|
||||
|
||||
### Day 29: 视频流水线优化 + CosyVoice 语气控制
|
||||
- [x] **字幕同步修复**: Whisper 时间戳三步平滑（单调递增+重叠消除+间隙填补）+ 原文节奏映射（线性插值 + 单字时长钳位）。
|
||||
- [x] **LatentSync 嘴型参数调优**: inference_steps 16→20, guidance_scale 2.0, DeepCache 启用, Remotion concurrency 16→4。
|
||||
- [x] **compose 流复制**: 不循环时 `-c:v copy` 替代 libx264 重编码，compose 耗时从分钟级降到秒级。
|
||||
- [x] **FFmpeg 超时保护**: `_run_ffmpeg()` timeout=600, `_get_duration()` timeout=30。
|
||||
- [x] **全局并发限制**: `asyncio.Semaphore(2)` 控制同时运行的生成任务数。
|
||||
- [x] **Redis 任务 TTL**: create 24h, completed/failed 2h, list 自动清理过期索引。
|
||||
- [x] **临时字体清理**: 字体文件加入 temp_files 清理列表。
|
||||
- [x] **预览背景 CORS 修复**: 素材同源代理 `/api/materials/stream/{id}` 彻底绕开跨域。
|
||||
- [x] **CosyVoice 语气控制**: 声音克隆模式新增语气下拉（正常/欢快/低沉/严肃），基于 `inference_instruct2()` 自然语言指令控制情绪，全链路透传 instruct_text，默认"正常"行为不变。
|
||||
|
||||
### Day 28: CosyVoice FP16 加速 + 文档全面更新
|
||||
- [x] **CosyVoice FP16 半精度加速**: `AutoModel()` 开启 `fp16=True`，LLM 推理和 Flow Matching 自动混合精度运行，预估提速 30-40%、显存降低 ~30%。
|
||||
- [x] **文档全面更新**: README.md / DEPLOY_MANUAL.md / SUBTITLE_DEPLOY.md / BACKEND_README.md 补充 MuseTalk 混合唇形同步方案、性能优化、Remotion 并发渲染等内容。
|
||||
|
||||
### Day 27: Remotion 描边修复 + 字体样式扩展 + 混合唇形同步 + 性能优化
|
||||
- [x] **描边渲染修复**: 标题/副标题/字幕从 `textShadow` 4 方向模拟改为 CSS 原生 `-webkit-text-stroke` + `paint-order: stroke fill`，修复描边过粗和副标题重影问题。
|
||||
- [x] **字体样式扩展**: 标题样式 4→12 个（+庞门正道/优设标题圆/阿里数黑体/文道潮黑/无界黑/厚底黑/寒蝉半圆体/欣意吉祥宋），字幕样式 4→8 个（+少女粉/清新绿/金色隶书/楷体红字）。
|
||||
- [x] **描边参数优化**: 所有预设 `stroke_size` 从 8 降至 4~5，配合原生描边视觉更干净。
|
||||
- [x] **TypeScript 类型修复**: Root.tsx `Composition` 泛型与 `calculateMetadata` 参数类型对齐；Video.tsx `VideoProps` 添加索引签名兼容 `Record<string, unknown>`；VideoLayer.tsx 移除 `OffthreadVideo` 不支持的 `loop` prop。
|
||||
- [x] **进度条文案还原**: 进度条从显示后端推送消息改回固定 `正在AI生成中...`。
|
||||
- [x] **MuseTalk 混合唇形同步**: 部署 MuseTalk 1.5 常驻服务 (GPU0, 端口 8011)，按音频时长自动路由（由 `LIPSYNC_DURATION_THRESHOLD` 控制；本仓库当前 `.env` 为 100）— 短视频走 LatentSync，长视频走 MuseTalk，MuseTalk 不可用时自动回退。
|
||||
- [x] **MuseTalk 推理性能优化**: server.py v2 重写 — cv2 直读帧(跳过 ffmpeg→PNG)、人脸检测降频(每5帧)、BiSeNet mask 缓存(每5帧)、cv2.VideoWriter 直写(跳过 PNG 写盘)、batch_size 8→32，预估 30min→8-10min (~3x)。
|
||||
- [x] **Remotion 并发渲染优化**: render.ts 新增 concurrency 参数，从默认 8 提升到 16 (56核 CPU)，预估 5min→2-3min。
|
||||
|
||||
### Day 26: 前端优化：板块合并 + 序号标题 + UI 精细化
|
||||
- [x] **板块合并**: 首页 9 个独立板块合并为 5 个主板块（配音方式+配音列表→三、配音；视频素材+时间轴→四、素材编辑；历史作品+作品预览→六、作品）。
|
||||
- [x] **中文序号标题**: 一~十编号（首页一~六，发布页七~十），移除所有 emoji 图标。
|
||||
- [x] **embedded 模式**: 6 个组件支持 `embedded` prop，嵌入时不渲染外层卡片/标题。
|
||||
- [x] **配音列表两行布局**: embedded 模式第 1 行语速+生成配音（右对齐），第 2 行配音列表+刷新。
|
||||
- [x] **子组件自渲染子标题**: MaterialSelector/TimelineEditor embedded 时自渲染 h3 子标题+操作按钮同行。
|
||||
- [x] **下拉对齐**: TitleSubtitlePanel 标签统一 `w-20`，下拉 `w-1/3 min-w-[100px]`，垂直对齐。
|
||||
- [x] **参考音频文案简化**: 底部段落移至标题旁，简化为 `(上传3-10秒语音样本)`。
|
||||
- [x] **账户手机号显示**: AccountSettingsDropdown 新增手机号显示。
|
||||
- [x] **标题显示模式对副标题生效**: payload 条件修复 + UI 下拉上移至板块标题行。
|
||||
- [x] **登录后用户信息立即可用**: AuthContext 暴露 `setUser`，登录成功后立即写入用户数据，修复登录后显示"未知账户"的问题。
|
||||
- [x] **文案微调**: 素材描述改为"上传自拍视频，最多可选4个"；显示模式选项加"标题"前缀。
|
||||
- [x] **UI/UX 体验优化**: 操作按钮移动端可见（opacity-40）、手机号脱敏、标题字数计数器、时间轴拖拽抓手图标、截取滑块放大。
|
||||
- [x] **代码质量修复**: 密码弹窗 success 清空、MaterialSelector useMemo + disabled 守卫、TimelineEditor useMemo。
|
||||
- [x] **发布页响应式布局**: 平台账号卡片单行布局，移动端紧凑（小图标/小按钮），桌面端宽松（与其他板块风格一致）。
|
||||
- [x] **移动端刷新回顶部**: `scrollRestoration = "manual"` + 列表 scroll 时间门控（`scrollEffectsEnabled` ref，1 秒内禁止自动滚动）+ 延迟兜底 `scrollTo(0,0)`。
|
||||
- [x] **移动端样式预览缩小**: FloatingStylePreview 移动端宽度缩至 160px，位置改为右下角，不遮挡样式调节控件。
|
||||
- [x] **列表滚动条统一隐藏**: 所有列表（BGM/配音/作品/素材/文案提取）滚动条改回 `hide-scrollbar`。
|
||||
- [x] **移动端配音/素材适配**: VoiceSelector 按钮移动端缩小（`px-2 sm:px-4`）修复克隆声音不可见；MaterialSelector 标题行移除 `whitespace-nowrap`，描述移动端隐藏，修复刷新按钮溢出。
|
||||
- [x] **生成配音按钮放大**: 从辅助尺寸（`text-xs px-2 py-1`）升级为主操作尺寸（`text-sm font-medium px-4 py-2`），新增阴影。
|
||||
- [x] **生成进度条位置调整**: 从"六、作品"卡片内部提取到右栏独立卡片，显示在作品卡片上方，更醒目。
|
||||
- [x] **LatentSync 超时修复**: httpx 超时从 1200s（20 分钟）改为 3600s（1 小时），修复 2 分钟以上视频口型推理超时回退问题。
|
||||
- [x] **字幕时间戳节奏映射**: `whisper_service.py` 从全程线性插值改为 Whisper 逐词节奏映射，修复长视频字幕漂移。
|
||||
|
||||
### Day 25: 文案提取修复 + 自定义提示词 + 片头副标题
|
||||
- [x] **抖音文案提取修复**: yt-dlp Fresh cookies 报错，重写 `_download_douyin_manual` 为移动端分享页 + 自动获取 ttwid 方案。
|
||||
- [x] **清理 DOUYIN_COOKIE**: 新方案不再需要手动维护 Cookie，从 `.env`/`config.py`/`service.py` 全面删除。
|
||||
- [x] **AI 智能改写自定义提示词**: 后端 `rewrite_script()` 支持 `custom_prompt` 参数；前端 checkbox 旁新增折叠式提示词编辑区，localStorage 持久化。
|
||||
- [x] **SSR 构建修复**: `useState` 初始化 `localStorage` 访问加 `typeof window` 守卫，修复 `npm run build` 报错。
|
||||
- [x] **片头副标题**: 新增 secondary_title（后端/Remotion/前端全链路），AI 同时生成，独立样式配置，20 字限制。
|
||||
- [x] **前端文案修正**: "AI 洗稿结果"→"AI 改写结果"。
|
||||
- [x] **yt-dlp 升级**: `2025.12.08` → `2026.2.21`。
|
||||
- [x] **参考音频中文文件名修复**: `sanitize_filename()` 将存储路径清洗为 ASCII 安全字符，纯中文名哈希兜底，原始名保留为展示名。
|
||||
|
||||
### Day 24: 鉴权到期治理 + 多素材时间轴稳定性修复
|
||||
- [x] **会员到期请求时失效**: 登录与鉴权接口统一执行 `expires_at` 检查；到期后自动停用账号、清理 session，并返回“会员已到期，请续费”。
|
||||
- [x] **画面比例控制**: 时间轴新增 `9:16 / 16:9` 输出比例选择，前端持久化并透传后端，单素材/多素材统一按目标分辨率处理。
|
||||
- [x] **标题/字幕防溢出**: Remotion 与前端预览统一响应式缩放、自动换行、描边/字距/边距比例缩放，降低预览与成片差异。
|
||||
- [x] **标题显示模式**: 标题行新增“短暂显示/常驻显示”下拉；默认短暂显示（4 秒），用户选择持久化并透传至 Remotion 渲染链路。
|
||||
- [x] **MOV 方向归一化**: 新增旋转元数据解析与 orientation normalize，修复“编码横屏+旋转元数据”导致的竖屏判断偏差。
|
||||
- [x] **多素材拼接稳定性**: 片段 prepare 与 concat 统一 25fps/CFR，concat 增加 `+genpts`，缓解段切换处“画面冻结口型还动”。
|
||||
- [x] **时间轴语义对齐**: 打通 `source_end` 全链路；修复 `sourceStart>0 且 sourceEnd=0` 时长计算；生成时以时间轴可见段 assignments 为准，超出段不参与。
|
||||
- [x] **交互细节优化**: 页面刷新回顶部；素材/历史列表首轮自动滚动抑制，减少恢复状态时页面跳动。
|
||||
|
||||
### Day 23: 配音前置重构 + 素材时间轴编排 + UI 体验优化 + 声音克隆增强
|
||||
|
||||
#### 第一阶段：配音前置
|
||||
- [x] **配音生成独立化**: 新增 `generated_audios` 后端模块（router/schemas/service），5 个 API 端点，复用现有 TTSService / voice_clone_service / task_store。
|
||||
- [x] **配音管理面板**: 前端新增 `useGeneratedAudios` hook + `GeneratedAudiosPanel` 组件，支持生成/试听/改名/删除/选中。
|
||||
- [x] **UI 面板重排序**: 文案 → 标题字幕 → 配音方式 → 配音列表 → 素材选择 → BGM → 生成视频。
|
||||
- [x] **素材区门控**: 未选中配音时素材区显示遮罩，选中后显示配音时长 + 素材均分信息。
|
||||
- [x] **视频生成对接**: workflow.py 新增预生成音频分支（`generated_audio_id`），跳过内联 TTS，向后兼容。
|
||||
- [x] **持久化**: selectedAudioId 加入 useHomePersistence，刷新页面恢复选中配音。
|
||||
|
||||
#### 第二阶段：素材时间轴编排
|
||||
- [x] **时间轴编辑器**: 新增 `TimelineEditor` 组件，wavesurfer.js 音频波形 + 色块可视化素材分配，拖拽分割线调整各段时长。
|
||||
- [x] **素材截取设置**: 新增 `ClipTrimmer` 模态框，HTML5 视频预览 + 双端滑块设置源视频截取起点/终点。
|
||||
- [x] **后端自定义分配**: 新增 `CustomAssignment` 模型，`prepare_segment` 支持 `source_start`，workflow 多素材/单素材流水线支持 `custom_assignments`。
|
||||
- [x] **循环截取修复**: `stream_loop + source_start` 改为两步处理（先裁剪再循环），确保从截取起点循环而非从视频 0s 开始。
|
||||
- [x] **MaterialSelector 精简**: 移除旧的时长信息栏和拖拽排序区（功能迁移到 TimelineEditor）。
|
||||
|
||||
#### 第三阶段：UI 体验优化 + TTS 稳定性
|
||||
- [x] **TTS SoX PATH 修复**: `run_qwen_tts.sh` export conda env bin 到 PATH (Qwen3-TTS 已停用，已被 CosyVoice 3.0 替换)。
|
||||
- [x] **TTS 显存管理**: 每次生成后 `torch.cuda.empty_cache()`，asyncio.to_thread 避免阻塞事件循环 (CosyVoice 沿用相同机制)。
|
||||
- [x] **配音列表按钮统一**: Play/Edit/Delete 按钮右侧同组 hover 显示，与 RefAudioPanel 一致，移除文案摘要。
|
||||
- [x] **素材区解除配音门控**: 移除 MaterialSelector 的 selectedAudio 遮罩，素材随时可上传管理。
|
||||
- [x] **时间轴拖拽排序**: TimelineEditor 色块支持 HTML5 Drag & Drop 调换素材顺序。
|
||||
- [x] **截取设置 Range Slider**: ClipTrimmer 改为单轨道双手柄（紫色起点+粉色终点），替换两个独立滑块。
|
||||
- [x] **截取设置视频预览**: 视频区域可播放/暂停，从 sourceStart 到 sourceEnd 自动停止，拖拽手柄时实时 seek。
|
||||
|
||||
#### 第四阶段：历史文案 + Bug 修复
|
||||
- [x] **历史文案保存与加载**: 新增 `useSavedScripts` hook，手动保存/加载/删除历史文案，独立 localStorage 持久化。
|
||||
- [x] **时间轴拖拽修复**: `reorderSegments` 从属性交换改为数组移动（splice），修复拖拽后时长不跟随素材的 Bug。
|
||||
- [x] **按钮视觉统一**: 文案编辑区 4 个按钮统一为固定高度 `h-7`，移除多余 `<span>` 嵌套。
|
||||
- [x] **底部栏调整**: "保存文案"按钮移至底部右侧，移除预计时长显示。
|
||||
|
||||
#### 第五阶段：字幕语言不匹配 + 视频比例错位修复
|
||||
- [x] **字幕用原文替换 Whisper 转录**: `align()` 新增 `original_text` 参数，字幕文字永远用配音保存的原始文案。
|
||||
- [x] **Remotion 动态视频尺寸**: `calculateMetadata` 从 props 读取真实尺寸，修复标题/字幕比例错位。
|
||||
- [x] **英文空格丢失修复**: `split_word_to_chars` 遇到空格时 flush buffer + pending_space 标记。
|
||||
|
||||
#### 第六阶段：参考音频自动转写 + 语速控制
|
||||
- [x] **Whisper 自动转写 ref_text**: 上传参考音频时自动调用 Whisper 转写内容作为 ref_text，不再使用前端固定文字。
|
||||
- [x] **参考音频自动截取**: 超过 10 秒自动在静音点截取（ffmpeg silencedetect），末尾 0.1 秒淡出避免截断爆音。
|
||||
- [x] **重新识别功能**: 新增 `POST /ref-audios/{id}/retranscribe` 端点 + 前端 RotateCw 按钮，旧音频可重新转写并截取。
|
||||
- [x] **语速控制**: 全链路 speed 参数（前端选择器 → 持久化 → 后端 → CosyVoice `inference_zero_shot(speed=)`），5 档：较慢(0.8)/稍慢(0.9)/正常(1.0)/稍快(1.1)/较快(1.2)。
|
||||
- [x] **缺少参考音频门控**: 声音克隆模式下未选参考音频时，生成配音按钮禁用 + 黄色警告提示。
|
||||
- [x] **Whisper 语言自动检测**: `transcribe()` language 参数改为可选（默认 None = 自动检测），支持多语言参考音频。
|
||||
- [x] **前端清理**: 移除固定 ref_text 常量、朗读引导文字，简化为"上传任意语音样本，系统将自动识别内容并克隆声音"。
|
||||
|
||||
### Day 22: 多素材优化 + AI 翻译 + TTS 多语言
|
||||
- [x] **多素材 Bug 修复**: 6 个高优 Bug（边界溢出、单段 fallback、除零、duration 校验、Whisper 兜底、空列表检查）。
|
||||
- [x] **架构重构**: 多素材从"逐段 LatentSync"重构为"先拼接再推理"，推理次数 N→1。
|
||||
- [x] **前端优化**: payload 安全、进度消息、上传自动选中、Material 接口统一、拖拽修复、素材上限 4 个。
|
||||
- [x] **AI 多语言翻译**: 新增 `/api/ai/translate` 接口，前端 9 种语言翻译 + 还原原文。
|
||||
- [x] **TTS 多语言**: EdgeTTS 10 语言声音列表、翻译自动切换声音、声音克隆 language 透传、textLang 持久化。
|
||||
|
||||
### Day 21: 缺陷修复 + 浮动预览 + 发布重构 + 架构优化 + 多素材生成
|
||||
- [x] **Remotion 崩溃容错**: 渲染进程 SIGABRT 退出时检查输出文件，避免误判失败导致标题/字幕丢失。
|
||||
- [x] **首页作品选择持久化**: 修复 `fetchGeneratedVideos` 无条件覆盖恢复值的问题，新增 `preferVideoId` 参数控制选中逻辑。
|
||||
- [x] **发布页作品选择持久化**: 根因为签名 URL 不稳定，全面改用 `video.id` 替代 `path` 进行选择/持久化/比较。
|
||||
- [x] **预取缓存补全**: 首页预取发布页数据时加入 `id` 字段，确保缓存数据可用于持久化匹配。
|
||||
- [x] **浮动样式预览窗口**: 标题字幕预览改为 `position: fixed` 浮动窗口，固定左上角，滚动时始终可见。
|
||||
- [x] **移动端适配**: ScriptEditor 按钮换行、预览默认比例改为 9:16 竖屏。
|
||||
- [x] **多平台发布重构**: 平台配置独立化（DOUYIN_*/WEIXIN_*）、用户隔离 Cookie 管理、抖音刷脸验证二维码、微信发布流程优化。
|
||||
- [x] **前端结构微调**: ScriptExtractionModal 迁移到 features/、contexts 迁移到 shared/contexts/、清理空目录。
|
||||
- [x] **后端模块分层**: materials/tools/ref_audios 三个模块补全 router+schemas+service 分层。
|
||||
- [x] **开发规范更新**: BACKEND_DEV.md 新增渐进原则、DOC_RULES.md 取消 TASK_COMPLETE.md 手动触发约束。
|
||||
- [x] **文档全面更新**: BACKEND_DEV/README、FRONTEND_DEV、DEPLOY_MANUAL、README.md 同步更新。
|
||||
- [x] **多素材视频生成（多机位效果）**: 支持多选素材 + 拖拽排序，按素材数量均分音频时长（对齐 Whisper 字边界）自动切换机位。逐段 LatentSync + FFmpeg 拼接。前端 @dnd-kit 拖拽排序 UI。
|
||||
- [x] **字幕开关移除**: 默认启用逐字高亮字幕，移除开关及相关死代码。
|
||||
- [x] **视频格式扩展**: 上传支持 mkv/webm/flv/wmv/m4v/ts/mts 等常见格式。
|
||||
- [x] **Watchdog 优化**: 健康检查阈值提高到 5 次，新增重启冷却期 120 秒，避免误重启。
|
||||
- [x] **多素材 Bug 修复**: 修复标点分句方案对无句末标点文案无效（改为均分方案）、音频时间偏移导致口型不对齐等缺陷。
|
||||
|
||||
### Day 20: 代码质量与安全优化
|
||||
- [x] **功能性修复**: LatentSync 回退逻辑、任务状态接口认证、User 类型统一。
|
||||
- [x] **性能优化**: N+1 查询修复、视频上传流式处理、httpx 异步替换、GLM 异步包装。
|
||||
- [x] **安全修复**: 硬编码 Cookie 配置化、日志敏感信息脱敏、ffprobe 安全调用、CORS 配置化。
|
||||
- [x] **配置优化**: 存储路径环境变量化、Remotion 预编译加速、LatentSync 绝对路径。
|
||||
- [x] **文档更新**: 更新 DOC_RULES.md 清单，补齐后端与部署文档；更新 SUBTITLE_DEPLOY.md, FRONTEND_DEV.md, implementation_plan.md。
|
||||
- [x] **缺陷修复**: 修复 Remotion 路径解析、发布页持久化竞态、首页选中回归、素材闭包陷阱。
|
||||
|
||||
### Day 19: 自动发布稳定性与发布体验优化 🚀
|
||||
- [x] **抖音发布稳定性**: 上传入口、封面流程、发布重试、登录失效识别与网络失败快速返回全面增强。
|
||||
- [x] **视频号发布修复**: 标题+标签统一写入“视频描述”，`post_create` 成功信号快速判定，超时改为失败返回。
|
||||
- [x] **成功截图闭环**: 抖音/视频号发布成功截图接入前端，支持用户隔离存储与鉴权访问。
|
||||
- [x] **截图观感优化**: 成功截图延后 3 秒并改为视口截图，修复“截图内容仅占 1/3”问题。
|
||||
- [x] **调试能力开关化**: 新增视频号录屏配置，默认可按环境变量开关，失败排障更直观。
|
||||
- [x] **启动链路统一**: 合并为 `run_backend.sh`（xvfb + headful），统一端口 `8006`，减少多进程混淆。
|
||||
- [x] **发布页防误操作**: 发布中按钮提示“请勿刷新或关闭网页”，并启用刷新/关页二次确认拦截。
|
||||
- [ ] **后续优化**: 发布任务状态恢复机制（任务化 + 状态持久化 + 前端轮询恢复）。
|
||||
|
||||
### Day 18: 后端模块化与规范完善
|
||||
- [x] **模块化迁移**: 路由透传 `modules/*`，业务逻辑集中到 service/workflow。
|
||||
- [x] **视频生成拆分**: 生成流程下沉 workflow，任务状态统一 TaskStore。
|
||||
- [x] **Redis 任务存储**: Redis 优先，不可用自动回退内存。
|
||||
- [x] **仓储层抽离**: Supabase 访问统一 `repositories/*`，deps/auth/admin 全面替换。
|
||||
- [x] **响应规范**: 统一 `success/message/data/code` + 全局异常处理。
|
||||
- [x] **素材重命名**: 新增重命名接口与 Storage `move_file`。
|
||||
- [x] **平台顺序调整**: 抖音/微信视频号/B站/小红书，移除快手。
|
||||
- [x] **后端开发规范**: 新增 `BACKEND_DEV.md`，README 同步模块化结构。
|
||||
- [x] **发布管理体验**: 首页预取路由 + 发布页骨架与缓存，进入更快。
|
||||
- [x] **素材加载优化**: 素材列表并发签名 URL，骨架数量动态。
|
||||
- [x] **预览加载优化**: `preload="metadata"` + hover 预取。
|
||||
|
||||
### Day 17: 前端重构与体验优化
|
||||
- [x] **UI 组件拆分**: 首页拆分为独立组件，降低 `page.tsx` 复杂度。
|
||||
- [x] **轻量 FSD 迁移**: `app` 页面轻量化，逻辑集中到 `features/*/model`，通用能力下沉 `shared/*`。
|
||||
- [x] **Controller Hooks**: Home/Publish 页面逻辑集中到 Controller Hook，Page 仅组合渲染。
|
||||
- [x] **通用工具抽取**: `media.ts` 统一 API Base / URL / 日期格式化。
|
||||
- [x] **交互优化**: 选择项持久化、列表内定位、刷新回顶部、最新作品优先预览。
|
||||
- [x] **发布页改造**: 作品列表卡片化 + 搜索 + 预览弹窗。
|
||||
- [x] **预览体验**: 预览弹窗统一头部样式与提示文案。
|
||||
- [x] **预览一致性**: 标题/字幕预览按素材分辨率缩放。
|
||||
- [x] **标题同步与限制**: 片头标题同步发布标题，输入法合成态兼容，限制 15 字。
|
||||
- [x] **样式默认与持久化**: 默认样式与字号调整，刷新保留用户选择。
|
||||
- [x] **性能微优化**: 列表渲染优化 + 并行请求 + localStorage 防抖。
|
||||
- [x] **资源能力**: 字体/BGM 资源库 + `/api/assets` 接入。
|
||||
- [x] **音频与字幕修复**: BGM 混音稳定性与字幕断句优化。
|
||||
- [x] **持久化修复**: 接入 `useHomePersistence`，恢复 `isRestored` 逻辑并通过构建。
|
||||
- [x] **预览与选择修复**: 发布预览兼容签名 URL，音频试听路径解析，素材/BGM 回退有效项。
|
||||
- [x] **体验细节优化**: 录音预览 URL 回收，预览弹窗滚动恢复，全局任务提示挂载。
|
||||
|
||||
### Day 16: 深度性能优化
|
||||
- [x] **Qwen-TTS 加速**: 集成 Flash Attention 2 (已停用，被 CosyVoice 3.0 替换)。
|
||||
- [x] **服务守护**: 开发 `Watchdog` 看门狗机制，自动监控并重启僵死服务。
|
||||
- [x] **LatentSync 性能确认**: 验证 DeepCache + 原生 Flash Attn 生效。
|
||||
- [x] **文档重构**: 全面更新 README、部署手册及后端文档。
|
||||
|
||||
### Day 15: 手机号认证迁移
|
||||
- [x] **认证系统升级**: 从邮箱迁移至 11 位手机号注册/登录。
|
||||
- [x] **账户管理**: 新增修改密码、有效期显示、安全退出功能。
|
||||
- [x] **AI 文案助手**: 升级 GLM-4.7-Flash，支持 B站/抖音链接提取与洗稿。
|
||||
|
||||
### Day 14: AI 增强与体验优化
|
||||
- [x] **AI 标题/标签**: 集成 GLM-4API 自动生成视频元数据。
|
||||
- [x] **字幕升级**: Remotion 逐字高亮字幕 (卡拉OK效果) 及动画片头。
|
||||
- [x] **模型升级**: 声音克隆已迁移至 CosyVoice 3.0 (0.5B)。
|
||||
|
||||
### Day 13: 声音克隆集成
|
||||
- [x] **声音克隆微服务**: 封装 CosyVoice 3.0 为独立 API (8010端口，替换 Qwen3-TTS)。
|
||||
- [x] **参考音频管理**: Supabase 存储桶配置与管理接口。
|
||||
- [x] **多模态 TTS**: 前端支持 EdgeTTS / Clone Voice 切换。
|
||||
|
||||
### Day 12: 移动端适配
|
||||
- [x] **iOS 兼容**: 修复 Safari 安全区域、状态栏颜色、Cookie 拦截问题。
|
||||
- [x] **响应式 UI**: 移动端 Header 与发布页重构。
|
||||
|
||||
### Day 11: 上传架构重构
|
||||
- [x] **直传优化**: 前端直传 Supabase Storage，解决 Nginx 30s 超时问题。
|
||||
- [x] **数据隔离**: 用户素材/视频按 UserID 物理隔离。
|
||||
|
||||
### Day 10: HTTPS 与安全
|
||||
- [x] **HTTPS 部署**: 配置 SSL 证书与 Nginx 反向代理。
|
||||
- [x] **安全加固**: Supabase Studio 增加 Basic Auth 保护。
|
||||
|
||||
### Day 9: 认证系统与发布闭环
|
||||
- [x] **用户系统**: 基于 Supabase Auth 实现 JWT 认证。
|
||||
- [x] **发布闭环**: 验证 B站/抖音/小红书 自动发布流程。
|
||||
- [x] **服务自愈**: 配置 PM2 进程守护。
|
||||
|
||||
### Day 1-8: 核心功能构建
|
||||
- [x] **Day 8**: 历史记录持久化与文件管理。
|
||||
- [x] **Day 7**: 社交媒体自动登录与多平台发布。
|
||||
- [x] **Day 6**: **LatentSync 1.6** 升级与服务器部署。
|
||||
- [x] **Day 5**: 前端视频上传与进度反馈。
|
||||
- [x] **Day 4**: MuseTalk (旧版) 口型同步修复。
|
||||
- [x] **Day 3**: 服务器环境配置与模型权重下载。
|
||||
- [x] **Day 1-2**: 项目基础框架 (FastAPI + Next.js) 搭建。
|
||||
|
||||
---
|
||||
|
||||
## 🛤️ 后续规划 (Roadmap)
|
||||
|
||||
### 🔴 优先待办
|
||||
- [x] ~~**配音前置重构 — 第二阶段**: 素材片段截取 + 语音时间轴编排~~ ✅ Day 23 已完成
|
||||
- [ ] **批量生成架构**: 支持 Excel 导入，批量生产视频。
|
||||
- [ ] **定时任务后台化**: 迁移前端触发的定时发布到后端 APScheduler。
|
||||
- [ ] **发布任务恢复机制**: 发布任务化 + 状态持久化 + 前端断点恢复，解决刷新后状态丢失。
|
||||
|
||||
### 🔵 长期探索
|
||||
- [ ] **容器化交付**: 提供完整的 Docker Compose 一键部署包。
|
||||
- [ ] **分布式队列**: 引入 Celery + Redis 处理超高并发任务。
|
||||
|
||||
---
|
||||
|
||||
## 📊 模块完成度
|
||||
|
||||
| 模块 | 进度 | 状态 |
|
||||
|------|------|------|
|
||||
| **核心 API** | 100% | ✅ 稳定 |
|
||||
| **Web UI** | 100% | ✅ 稳定 (移动端适配) |
|
||||
| **唇形同步** | 100% | ✅ LatentSync 1.6 |
|
||||
| **TTS 配音** | 100% | ✅ EdgeTTS + CosyVoice 3.0 + 配音前置 + 时间轴编排 + 自动转写 + 语速控制 + 语气控制 |
|
||||
| **自动发布** | 100% | ✅ 抖音/微信视频号/B站/小红书 |
|
||||
| **用户认证** | 100% | ✅ 手机号 + JWT |
|
||||
| **付费会员** | 100% | ✅ 支付宝电脑网站支付 + 自动激活 |
|
||||
| **部署运维** | 100% | ✅ PM2 + Watchdog |
|
||||
|
||||
---
|
||||
|
||||
## 📎 相关文档
|
||||
|
||||
- [详细开发日志 (DevLogs)](Docs/DevLogs/)
|
||||
- [部署手册 (DEPLOY_MANUAL)](Docs/DEPLOY_MANUAL.md)
|
||||
- [x] **MuseTalk 推理性能优化**: server.py v2 重写 — cv2 直读帧(跳过 ffmpeg→PNG)、人脸检测降频(每5帧)、BiSeNet mask 缓存(每5帧)、cv2.VideoWriter 直写(跳过 PNG 写盘)、batch_size 8→32，预估 30min→8-10min (~3x)。
|
||||
- [x] **Remotion 并发渲染优化**: render.ts 新增 concurrency 参数，从默认 8 提升到 16 (56核 CPU)，预估 5min→2-3min。
|
||||
|
||||
### Day 26: 前端优化：板块合并 + 序号标题 + UI 精细化
|
||||
- [x] **板块合并**: 首页 9 个独立板块合并为 5 个主板块（配音方式+配音列表→三、配音；视频素材+时间轴→四、素材编辑；历史作品+作品预览→六、作品）。
|
||||
- [x] **中文序号标题**: 一~十编号（首页一~六，发布页七~十），移除所有 emoji 图标。
|
||||
- [x] **embedded 模式**: 6 个组件支持 `embedded` prop，嵌入时不渲染外层卡片/标题。
|
||||
- [x] **配音列表两行布局**: embedded 模式第 1 行语速+生成配音（右对齐），第 2 行配音列表+刷新。
|
||||
- [x] **子组件自渲染子标题**: MaterialSelector/TimelineEditor embedded 时自渲染 h3 子标题+操作按钮同行。
|
||||
- [x] **下拉对齐**: TitleSubtitlePanel 标签统一 `w-20`，下拉 `w-1/3 min-w-[100px]`，垂直对齐。
|
||||
- [x] **参考音频文案简化**: 底部段落移至标题旁，简化为 `(上传3-10秒语音样本)`。
|
||||
- [x] **账户手机号显示**: AccountSettingsDropdown 新增手机号显示。
|
||||
- [x] **标题显示模式对副标题生效**: payload 条件修复 + UI 下拉上移至板块标题行。
|
||||
- [x] **登录后用户信息立即可用**: AuthContext 暴露 `setUser`，登录成功后立即写入用户数据，修复登录后显示"未知账户"的问题。
|
||||
- [x] **文案微调**: 素材描述改为"上传自拍视频，最多可选4个"；显示模式选项加"标题"前缀。
|
||||
- [x] **UI/UX 体验优化**: 操作按钮移动端可见（opacity-40）、手机号脱敏、标题字数计数器、时间轴拖拽抓手图标、截取滑块放大。
|
||||
- [x] **代码质量修复**: 密码弹窗 success 清空、MaterialSelector useMemo + disabled 守卫、TimelineEditor useMemo。
|
||||
- [x] **发布页响应式布局**: 平台账号卡片单行布局，移动端紧凑（小图标/小按钮），桌面端宽松（与其他板块风格一致）。
|
||||
- [x] **移动端刷新回顶部**: `scrollRestoration = "manual"` + 列表 scroll 时间门控（`scrollEffectsEnabled` ref，1 秒内禁止自动滚动）+ 延迟兜底 `scrollTo(0,0)`。
|
||||
- [x] **移动端样式预览缩小**: FloatingStylePreview 移动端宽度缩至 160px，位置改为右下角，不遮挡样式调节控件。
|
||||
- [x] **列表滚动条统一隐藏**: 所有列表（BGM/配音/作品/素材/文案提取）滚动条改回 `hide-scrollbar`。
|
||||
- [x] **移动端配音/素材适配**: VoiceSelector 按钮移动端缩小（`px-2 sm:px-4`）修复克隆声音不可见；MaterialSelector 标题行移除 `whitespace-nowrap`，描述移动端隐藏，修复刷新按钮溢出。
|
||||
- [x] **生成配音按钮放大**: 从辅助尺寸（`text-xs px-2 py-1`）升级为主操作尺寸（`text-sm font-medium px-4 py-2`），新增阴影。
|
||||
- [x] **生成进度条位置调整**: 从"六、作品"卡片内部提取到右栏独立卡片，显示在作品卡片上方，更醒目。
|
||||
- [x] **LatentSync 超时修复**: httpx 超时从 1200s（20 分钟）改为 3600s（1 小时），修复 2 分钟以上视频口型推理超时回退问题。
|
||||
- [x] **字幕时间戳节奏映射**: `whisper_service.py` 从全程线性插值改为 Whisper 逐词节奏映射，修复长视频字幕漂移。
|
||||
|
||||
### Day 25: 文案提取修复 + 自定义提示词 + 片头副标题
|
||||
- [x] **抖音文案提取修复**: yt-dlp Fresh cookies 报错，重写 `_download_douyin_manual` 为移动端分享页 + 自动获取 ttwid 方案。
|
||||
- [x] **清理 DOUYIN_COOKIE**: 新方案不再需要手动维护 Cookie，从 `.env`/`config.py`/`service.py` 全面删除。
|
||||
- [x] **AI 智能改写自定义提示词**: 后端 `rewrite_script()` 支持 `custom_prompt` 参数；前端 checkbox 旁新增折叠式提示词编辑区，localStorage 持久化。
|
||||
- [x] **SSR 构建修复**: `useState` 初始化 `localStorage` 访问加 `typeof window` 守卫，修复 `npm run build` 报错。
|
||||
- [x] **片头副标题**: 新增 secondary_title（后端/Remotion/前端全链路），AI 同时生成，独立样式配置，20 字限制。
|
||||
- [x] **前端文案修正**: "AI 洗稿结果"→"AI 改写结果"。
|
||||
- [x] **yt-dlp 升级**: `2025.12.08` → `2026.2.21`。
|
||||
- [x] **参考音频中文文件名修复**: `sanitize_filename()` 将存储路径清洗为 ASCII 安全字符，纯中文名哈希兜底，原始名保留为展示名。
|
||||
|
||||
### Day 24: 鉴权到期治理 + 多素材时间轴稳定性修复
|
||||
- [x] **会员到期请求时失效**: 登录与鉴权接口统一执行 `expires_at` 检查；到期后自动停用账号、清理 session，并返回“会员已到期，请续费”。
|
||||
- [x] **画面比例控制**: 时间轴新增 `9:16 / 16:9` 输出比例选择，前端持久化并透传后端，单素材/多素材统一按目标分辨率处理。
|
||||
- [x] **标题/字幕防溢出**: Remotion 与前端预览统一响应式缩放、自动换行、描边/字距/边距比例缩放，降低预览与成片差异。
|
||||
- [x] **标题显示模式**: 标题行新增“短暂显示/常驻显示”下拉；默认短暂显示（4 秒），用户选择持久化并透传至 Remotion 渲染链路。
|
||||
- [x] **MOV 方向归一化**: 新增旋转元数据解析与 orientation normalize，修复“编码横屏+旋转元数据”导致的竖屏判断偏差。
|
||||
- [x] **多素材拼接稳定性**: 片段 prepare 与 concat 统一 25fps/CFR，concat 增加 `+genpts`，缓解段切换处“画面冻结口型还动”。
|
||||
- [x] **时间轴语义对齐**: 打通 `source_end` 全链路；修复 `sourceStart>0 且 sourceEnd=0` 时长计算；生成时以时间轴可见段 assignments 为准，超出段不参与。
|
||||
- [x] **交互细节优化**: 页面刷新回顶部；素材/历史列表首轮自动滚动抑制，减少恢复状态时页面跳动。
|
||||
|
||||
### Day 23: 配音前置重构 + 素材时间轴编排 + UI 体验优化 + 声音克隆增强
|
||||
|
||||
#### 第一阶段：配音前置
|
||||
- [x] **配音生成独立化**: 新增 `generated_audios` 后端模块（router/schemas/service），5 个 API 端点，复用现有 TTSService / voice_clone_service / task_store。
|
||||
- [x] **配音管理面板**: 前端新增 `useGeneratedAudios` hook + `GeneratedAudiosPanel` 组件，支持生成/试听/改名/删除/选中。
|
||||
- [x] **UI 面板重排序**: 文案 → 标题字幕 → 配音方式 → 配音列表 → 素材选择 → BGM → 生成视频。
|
||||
- [x] **素材区门控**: 未选中配音时素材区显示遮罩，选中后显示配音时长 + 素材均分信息。
|
||||
- [x] **视频生成对接**: workflow.py 新增预生成音频分支（`generated_audio_id`），跳过内联 TTS，向后兼容。
|
||||
- [x] **持久化**: selectedAudioId 加入 useHomePersistence，刷新页面恢复选中配音。
|
||||
|
||||
#### 第二阶段：素材时间轴编排
|
||||
- [x] **时间轴编辑器**: 新增 `TimelineEditor` 组件，wavesurfer.js 音频波形 + 色块可视化素材分配，拖拽分割线调整各段时长。
|
||||
- [x] **素材截取设置**: 新增 `ClipTrimmer` 模态框，HTML5 视频预览 + 双端滑块设置源视频截取起点/终点。
|
||||
- [x] **后端自定义分配**: 新增 `CustomAssignment` 模型，`prepare_segment` 支持 `source_start`，workflow 多素材/单素材流水线支持 `custom_assignments`。
|
||||
- [x] **循环截取修复**: `stream_loop + source_start` 改为两步处理（先裁剪再循环），确保从截取起点循环而非从视频 0s 开始。
|
||||
- [x] **MaterialSelector 精简**: 移除旧的时长信息栏和拖拽排序区（功能迁移到 TimelineEditor）。
|
||||
|
||||
#### 第三阶段：UI 体验优化 + TTS 稳定性
|
||||
- [x] **TTS SoX PATH 修复**: `run_qwen_tts.sh` export conda env bin 到 PATH (Qwen3-TTS 已停用，已被 CosyVoice 3.0 替换)。
|
||||
- [x] **TTS 显存管理**: 每次生成后 `torch.cuda.empty_cache()`，asyncio.to_thread 避免阻塞事件循环 (CosyVoice 沿用相同机制)。
|
||||
- [x] **配音列表按钮统一**: Play/Edit/Delete 按钮右侧同组 hover 显示，与 RefAudioPanel 一致，移除文案摘要。
|
||||
- [x] **素材区解除配音门控**: 移除 MaterialSelector 的 selectedAudio 遮罩，素材随时可上传管理。
|
||||
- [x] **时间轴拖拽排序**: TimelineEditor 色块支持 HTML5 Drag & Drop 调换素材顺序。
|
||||
- [x] **截取设置 Range Slider**: ClipTrimmer 改为单轨道双手柄（紫色起点+粉色终点），替换两个独立滑块。
|
||||
- [x] **截取设置视频预览**: 视频区域可播放/暂停，从 sourceStart 到 sourceEnd 自动停止，拖拽手柄时实时 seek。
|
||||
|
||||
#### 第四阶段：历史文案 + Bug 修复
|
||||
- [x] **历史文案保存与加载**: 新增 `useSavedScripts` hook，手动保存/加载/删除历史文案，独立 localStorage 持久化。
|
||||
- [x] **时间轴拖拽修复**: `reorderSegments` 从属性交换改为数组移动（splice），修复拖拽后时长不跟随素材的 Bug。
|
||||
- [x] **按钮视觉统一**: 文案编辑区 4 个按钮统一为固定高度 `h-7`，移除多余 `<span>` 嵌套。
|
||||
- [x] **底部栏调整**: "保存文案"按钮移至底部右侧，移除预计时长显示。
|
||||
|
||||
#### 第五阶段：字幕语言不匹配 + 视频比例错位修复
|
||||
- [x] **字幕用原文替换 Whisper 转录**: `align()` 新增 `original_text` 参数，字幕文字永远用配音保存的原始文案。
|
||||
- [x] **Remotion 动态视频尺寸**: `calculateMetadata` 从 props 读取真实尺寸，修复标题/字幕比例错位。
|
||||
- [x] **英文空格丢失修复**: `split_word_to_chars` 遇到空格时 flush buffer + pending_space 标记。
|
||||
|
||||
#### 第六阶段：参考音频自动转写 + 语速控制
|
||||
- [x] **Whisper 自动转写 ref_text**: 上传参考音频时自动调用 Whisper 转写内容作为 ref_text，不再使用前端固定文字。
|
||||
- [x] **参考音频自动截取**: 超过 10 秒自动在静音点截取（ffmpeg silencedetect），末尾 0.1 秒淡出避免截断爆音。
|
||||
- [x] **重新识别功能**: 新增 `POST /ref-audios/{id}/retranscribe` 端点 + 前端 RotateCw 按钮，旧音频可重新转写并截取。
|
||||
- [x] **语速控制**: 全链路 speed 参数（前端选择器 → 持久化 → 后端 → CosyVoice `inference_zero_shot(speed=)`），5 档：较慢(0.8)/稍慢(0.9)/正常(1.0)/稍快(1.1)/较快(1.2)。
|
||||
- [x] **缺少参考音频门控**: 声音克隆模式下未选参考音频时，生成配音按钮禁用 + 黄色警告提示。
|
||||
- [x] **Whisper 语言自动检测**: `transcribe()` language 参数改为可选（默认 None = 自动检测），支持多语言参考音频。
|
||||
- [x] **前端清理**: 移除固定 ref_text 常量、朗读引导文字，简化为"上传任意语音样本，系统将自动识别内容并克隆声音"。
|
||||
|
||||
### Day 22: 多素材优化 + AI 翻译 + TTS 多语言
|
||||
- [x] **多素材 Bug 修复**: 6 个高优 Bug（边界溢出、单段 fallback、除零、duration 校验、Whisper 兜底、空列表检查）。
|
||||
- [x] **架构重构**: 多素材从"逐段 LatentSync"重构为"先拼接再推理"，推理次数 N→1。
|
||||
- [x] **前端优化**: payload 安全、进度消息、上传自动选中、Material 接口统一、拖拽修复、素材上限 4 个。
|
||||
- [x] **AI 多语言翻译**: 新增 `/api/ai/translate` 接口，前端 9 种语言翻译 + 还原原文。
|
||||
- [x] **TTS 多语言**: EdgeTTS 10 语言声音列表、翻译自动切换声音、声音克隆 language 透传、textLang 持久化。
|
||||
|
||||
### Day 21: 缺陷修复 + 浮动预览 + 发布重构 + 架构优化 + 多素材生成
|
||||
- [x] **Remotion 崩溃容错**: 渲染进程 SIGABRT 退出时检查输出文件，避免误判失败导致标题/字幕丢失。
|
||||
- [x] **首页作品选择持久化**: 修复 `fetchGeneratedVideos` 无条件覆盖恢复值的问题，新增 `preferVideoId` 参数控制选中逻辑。
|
||||
- [x] **发布页作品选择持久化**: 根因为签名 URL 不稳定，全面改用 `video.id` 替代 `path` 进行选择/持久化/比较。
|
||||
- [x] **预取缓存补全**: 首页预取发布页数据时加入 `id` 字段，确保缓存数据可用于持久化匹配。
|
||||
- [x] **浮动样式预览窗口**: 标题字幕预览改为 `position: fixed` 浮动窗口，固定左上角，滚动时始终可见。
|
||||
- [x] **移动端适配**: ScriptEditor 按钮换行、预览默认比例改为 9:16 竖屏。
|
||||
- [x] **多平台发布重构**: 平台配置独立化（DOUYIN_*/WEIXIN_*）、用户隔离 Cookie 管理、抖音刷脸验证二维码、微信发布流程优化。
|
||||
- [x] **前端结构微调**: ScriptExtractionModal 迁移到 features/、contexts 迁移到 shared/contexts/、清理空目录。
|
||||
- [x] **后端模块分层**: materials/tools/ref_audios 三个模块补全 router+schemas+service 分层。
|
||||
- [x] **开发规范更新**: BACKEND_DEV.md 新增渐进原则、DOC_RULES.md 取消 TASK_COMPLETE.md 手动触发约束。
|
||||
- [x] **文档全面更新**: BACKEND_DEV/README、FRONTEND_DEV、DEPLOY_MANUAL、README.md 同步更新。
|
||||
- [x] **多素材视频生成（多机位效果）**: 支持多选素材 + 拖拽排序，按素材数量均分音频时长（对齐 Whisper 字边界）自动切换机位。逐段 LatentSync + FFmpeg 拼接。前端 @dnd-kit 拖拽排序 UI。
|
||||
- [x] **字幕开关移除**: 默认启用逐字高亮字幕，移除开关及相关死代码。
|
||||
- [x] **视频格式扩展**: 上传支持 mkv/webm/flv/wmv/m4v/ts/mts 等常见格式。
|
||||
- [x] **Watchdog 优化**: 健康检查阈值提高到 5 次，新增重启冷却期 120 秒，避免误重启。
|
||||
- [x] **多素材 Bug 修复**: 修复标点分句方案对无句末标点文案无效（改为均分方案）、音频时间偏移导致口型不对齐等缺陷。
|
||||
|
||||
### Day 20: 代码质量与安全优化
|
||||
- [x] **功能性修复**: LatentSync 回退逻辑、任务状态接口认证、User 类型统一。
|
||||
- [x] **性能优化**: N+1 查询修复、视频上传流式处理、httpx 异步替换、GLM 异步包装。
|
||||
- [x] **安全修复**: 硬编码 Cookie 配置化、日志敏感信息脱敏、ffprobe 安全调用、CORS 配置化。
|
||||
- [x] **配置优化**: 存储路径环境变量化、Remotion 预编译加速、LatentSync 绝对路径。
|
||||
- [x] **文档更新**: 更新 DOC_RULES.md 清单，补齐后端与部署文档；更新 SUBTITLE_DEPLOY.md, FRONTEND_DEV.md, implementation_plan.md。
|
||||
- [x] **缺陷修复**: 修复 Remotion 路径解析、发布页持久化竞态、首页选中回归、素材闭包陷阱。
|
||||
|
||||
### Day 19: 自动发布稳定性与发布体验优化 🚀
|
||||
- [x] **抖音发布稳定性**: 上传入口、封面流程、发布重试、登录失效识别与网络失败快速返回全面增强。
|
||||
- [x] **视频号发布修复**: 标题+标签统一写入“视频描述”，`post_create` 成功信号快速判定，超时改为失败返回。
|
||||
- [x] **成功截图闭环**: 抖音/视频号发布成功截图接入前端，支持用户隔离存储与鉴权访问。
|
||||
- [x] **截图观感优化**: 成功截图延后 3 秒并改为视口截图，修复“截图内容仅占 1/3”问题。
|
||||
- [x] **调试能力开关化**: 新增视频号录屏配置，默认可按环境变量开关，失败排障更直观。
|
||||
- [x] **启动链路统一**: 合并为 `run_backend.sh`（xvfb + headful），统一端口 `8006`，减少多进程混淆。
|
||||
- [x] **发布页防误操作**: 发布中按钮提示“请勿刷新或关闭网页”，并启用刷新/关页二次确认拦截。
|
||||
- [ ] **后续优化**: 发布任务状态恢复机制（任务化 + 状态持久化 + 前端轮询恢复）。
|
||||
|
||||
### Day 18: 后端模块化与规范完善
|
||||
- [x] **模块化迁移**: 路由透传 `modules/*`，业务逻辑集中到 service/workflow。
|
||||
- [x] **视频生成拆分**: 生成流程下沉 workflow，任务状态统一 TaskStore。
|
||||
- [x] **Redis 任务存储**: Redis 优先，不可用自动回退内存。
|
||||
- [x] **仓储层抽离**: Supabase 访问统一 `repositories/*`，deps/auth/admin 全面替换。
|
||||
- [x] **响应规范**: 统一 `success/message/data/code` + 全局异常处理。
|
||||
- [x] **素材重命名**: 新增重命名接口与 Storage `move_file`。
|
||||
- [x] **平台顺序调整**: 抖音/微信视频号/B站/小红书，移除快手。
|
||||
- [x] **后端开发规范**: 新增 `BACKEND_DEV.md`，README 同步模块化结构。
|
||||
- [x] **发布管理体验**: 首页预取路由 + 发布页骨架与缓存，进入更快。
|
||||
- [x] **素材加载优化**: 素材列表并发签名 URL，骨架数量动态。
|
||||
- [x] **预览加载优化**: `preload="metadata"` + hover 预取。
|
||||
|
||||
### Day 17: 前端重构与体验优化
|
||||
- [x] **UI 组件拆分**: 首页拆分为独立组件，降低 `page.tsx` 复杂度。
|
||||
- [x] **轻量 FSD 迁移**: `app` 页面轻量化，逻辑集中到 `features/*/model`，通用能力下沉 `shared/*`。
|
||||
- [x] **Controller Hooks**: Home/Publish 页面逻辑集中到 Controller Hook，Page 仅组合渲染。
|
||||
- [x] **通用工具抽取**: `media.ts` 统一 API Base / URL / 日期格式化。
|
||||
- [x] **交互优化**: 选择项持久化、列表内定位、刷新回顶部、最新作品优先预览。
|
||||
- [x] **发布页改造**: 作品列表卡片化 + 搜索 + 预览弹窗。
|
||||
- [x] **预览体验**: 预览弹窗统一头部样式与提示文案。
|
||||
- [x] **预览一致性**: 标题/字幕预览按素材分辨率缩放。
|
||||
- [x] **标题同步与限制**: 片头标题同步发布标题，输入法合成态兼容，限制 15 字。
|
||||
- [x] **样式默认与持久化**: 默认样式与字号调整，刷新保留用户选择。
|
||||
- [x] **性能微优化**: 列表渲染优化 + 并行请求 + localStorage 防抖。
|
||||
- [x] **资源能力**: 字体/BGM 资源库 + `/api/assets` 接入。
|
||||
- [x] **音频与字幕修复**: BGM 混音稳定性与字幕断句优化。
|
||||
- [x] **持久化修复**: 接入 `useHomePersistence`，恢复 `isRestored` 逻辑并通过构建。
|
||||
- [x] **预览与选择修复**: 发布预览兼容签名 URL，音频试听路径解析，素材/BGM 回退有效项。
|
||||
- [x] **体验细节优化**: 录音预览 URL 回收，预览弹窗滚动恢复，全局任务提示挂载。
|
||||
|
||||
### Day 16: 深度性能优化
|
||||
- [x] **Qwen-TTS 加速**: 集成 Flash Attention 2 (已停用，被 CosyVoice 3.0 替换)。
|
||||
- [x] **服务守护**: 开发 `Watchdog` 看门狗机制，自动监控并重启僵死服务。
|
||||
- [x] **LatentSync 性能确认**: 验证 DeepCache + 原生 Flash Attn 生效。
|
||||
- [x] **文档重构**: 全面更新 README、部署手册及后端文档。
|
||||
|
||||
### Day 15: 手机号认证迁移
|
||||
- [x] **认证系统升级**: 从邮箱迁移至 11 位手机号注册/登录。
|
||||
- [x] **账户管理**: 新增修改密码、有效期显示、安全退出功能。
|
||||
- [x] **AI 文案助手**: 升级 GLM-4.7-Flash，支持 B站/抖音链接提取与洗稿。
|
||||
|
||||
### Day 14: AI 增强与体验优化
|
||||
- [x] **AI 标题/标签**: 集成 GLM-4API 自动生成视频元数据。
|
||||
- [x] **字幕升级**: Remotion 逐字高亮字幕 (卡拉OK效果) 及动画片头。
|
||||
- [x] **模型升级**: 声音克隆已迁移至 CosyVoice 3.0 (0.5B)。
|
||||
|
||||
### Day 13: 声音克隆集成
|
||||
- [x] **声音克隆微服务**: 封装 CosyVoice 3.0 为独立 API (8010端口，替换 Qwen3-TTS)。
|
||||
- [x] **参考音频管理**: Supabase 存储桶配置与管理接口。
|
||||
- [x] **多模态 TTS**: 前端支持 EdgeTTS / Clone Voice 切换。
|
||||
|
||||
### Day 12: 移动端适配
|
||||
- [x] **iOS 兼容**: 修复 Safari 安全区域、状态栏颜色、Cookie 拦截问题。
|
||||
- [x] **响应式 UI**: 移动端 Header 与发布页重构。
|
||||
|
||||
### Day 11: 上传架构重构
|
||||
- [x] **直传优化**: 前端直传 Supabase Storage，解决 Nginx 30s 超时问题。
|
||||
- [x] **数据隔离**: 用户素材/视频按 UserID 物理隔离。
|
||||
|
||||
### Day 10: HTTPS 与安全
|
||||
- [x] **HTTPS 部署**: 配置 SSL 证书与 Nginx 反向代理。
|
||||
- [x] **安全加固**: Supabase Studio 增加 Basic Auth 保护。
|
||||
|
||||
### Day 9: 认证系统与发布闭环
|
||||
- [x] **用户系统**: 基于 Supabase Auth 实现 JWT 认证。
|
||||
- [x] **发布闭环**: 验证 B站/抖音/小红书 自动发布流程。
|
||||
- [x] **服务自愈**: 配置 PM2 进程守护。
|
||||
|
||||
### Day 1-8: 核心功能构建
|
||||
- [x] **Day 8**: 历史记录持久化与文件管理。
|
||||
- [x] **Day 7**: 社交媒体自动登录与多平台发布。
|
||||
- [x] **Day 6**: **LatentSync 1.6** 升级与服务器部署。
|
||||
- [x] **Day 5**: 前端视频上传与进度反馈。
|
||||
- [x] **Day 4**: MuseTalk (旧版) 口型同步修复。
|
||||
- [x] **Day 3**: 服务器环境配置与模型权重下载。
|
||||
- [x] **Day 1-2**: 项目基础框架 (FastAPI + Next.js) 搭建。
|
||||
|
||||
---
|
||||
|
||||
## 🛤️ 后续规划 (Roadmap)
|
||||
|
||||
### 🔴 优先待办
|
||||
- [x] ~~**配音前置重构 — 第二阶段**: 素材片段截取 + 语音时间轴编排~~ ✅ Day 23 已完成
|
||||
- [ ] **批量生成架构**: 支持 Excel 导入，批量生产视频。
|
||||
- [ ] **定时任务后台化**: 迁移前端触发的定时发布到后端 APScheduler。
|
||||
- [ ] **发布任务恢复机制**: 发布任务化 + 状态持久化 + 前端断点恢复，解决刷新后状态丢失。
|
||||
|
||||
### 🔵 长期探索
|
||||
- [ ] **容器化交付**: 提供完整的 Docker Compose 一键部署包。
|
||||
- [ ] **分布式队列**: 引入 Celery + Redis 处理超高并发任务。
|
||||
|
||||
---
|
||||
|
||||
## 📊 模块完成度
|
||||
|
||||
| 模块 | 进度 | 状态 |
|
||||
|------|------|------|
|
||||
| **核心 API** | 100% | ✅ 稳定 |
|
||||
| **Web UI** | 100% | ✅ 稳定 (移动端适配) |
|
||||
| **唇形同步** | 100% | ✅ LatentSync 1.6 |
|
||||
| **TTS 配音** | 100% | ✅ EdgeTTS + CosyVoice 3.0 + 配音前置 + 时间轴编排 + 自动转写 + 语速控制 + 语气控制 |
|
||||
| **自动发布** | 100% | ✅ 抖音/微信视频号/B站/小红书 |
|
||||
| **用户认证** | 100% | ✅ 手机号 + JWT |
|
||||
| **付费会员** | 100% | ✅ 支付宝电脑网站支付 + 自动激活 |
|
||||
| **部署运维** | 100% | ✅ PM2 + Watchdog |
|
||||
|
||||
---
|
||||
|
||||
## 📎 相关文档
|
||||
|
||||
- [详细开发日志 (DevLogs)](Docs/DevLogs/)
|
||||
- [部署手册 (DEPLOY_MANUAL)](Docs/DEPLOY_MANUAL.md)
|
||||
|
||||
36
README.md
36
README.md
@@ -16,31 +16,31 @@
|
||||
## ✨ 功能特性
|
||||
|
||||
### 核心能力
|
||||
- 🎬 **高清唇形同步** - 混合方案：短视频（本仓库当前 `.env` 阈值 100s，可配）用 LatentSync 1.6（高质量 Latent Diffusion），长视频用 MuseTalk 1.5（实时级单步推理），自动路由 + 回退。前端可选模型：默认模型（阈值自动路由）/ 快速模型（速度优先）/ 高级模型（质量优先）。
|
||||
- 🎬 **高清唇形同步** - 混合方案：短视频（本仓库当前 `.env` 阈值 100s，可配）用 LatentSync 1.6（高质量 Latent Diffusion），长视频用 MuseTalk 1.5（实时级单步推理），自动路由 + 回退。前端可选模型：默认模型（阈值自动路由）/ 快速模型（速度优先）/ 高级模型（质量优先）。
|
||||
- 🎙️ **多模态配音** - 支持 **EdgeTTS** (微软超自然语音, 10 语言) 和 **CosyVoice 3.0** (3秒极速声音克隆, 9语言+18方言, 语速/语气可调)。上传参考音频自动 Whisper 转写 + 智能截取。配音前置工作流：先生成配音 → 选素材 → 生成视频。
|
||||
- 📝 **智能字幕** - 集成 faster-whisper + Remotion，自动生成逐字高亮 (卡拉OK效果) 字幕。
|
||||
- 🎨 **样式预设** - 12 种标题 + 8 种字幕样式预设，支持预览 + 字号调节 + 自定义字体库。CSS 原生描边渲染，清晰无重影。
|
||||
- 🏷️ **标题显示模式** - 片头标题支持 `短暂显示` / `常驻显示`，默认短暂显示（4秒），用户偏好自动持久化。
|
||||
- 📌 **片头副标题** - 可选副标题显示在主标题下方，独立样式配置，AI 可同时生成，20 字限制。
|
||||
- 🖼️ **作品预览一致性** - 标题/字幕预览与 Remotion 成片统一响应式缩放和自动换行，窄屏画布也稳定显示。
|
||||
- 🎞️ **多素材多机位** - 支持多选素材 + 时间轴编辑器 (wavesurfer.js 波形可视化)，拖拽分割线调整时长、拖拽排序切换机位、按 `source_start/source_end` 截取片段。
|
||||
- 🎞️ **多素材多机位** - 支持多选素材 + 时间轴编辑器 (wavesurfer.js 波形可视化)，主素材连续循环播放 + 浮动插入镜头块自由叠加，拖拽移动位置、ClipTrimmer 统一编辑截取范围与时长，支持"设为主素材"切换。
|
||||
- 📐 **画面比例控制** - 时间轴一键切换 `9:16 / 16:9` 输出比例，生成链路全程按目标比例处理。
|
||||
- 💾 **用户偏好持久化** - 首页状态统一恢复/保存，刷新后延续上次配置；新作品生成后优先选中最新，后续用户手动选择持续持久化。
|
||||
- 🎵 **背景音乐** - 试听 + 搜索选择 + 混音（当前前端固定混音系数，保持配音音量稳定）。
|
||||
- 🧩 **统一选择器交互** - 首页/发布页业务选择项统一 SelectPopover（桌面 Popover / 移动端 BottomSheet），支持自动上拉、已选定位与连续预览。
|
||||
- 🤖 **AI 辅助创作** - 内置 GLM-4.7-Flash，支持 B站/抖音链接文案提取、AI 智能改写（支持自定义提示词）、标题/标签自动生成、9 语言翻译。
|
||||
- 💾 **用户偏好持久化** - 首页状态统一恢复/保存，刷新后延续上次配置；新作品生成后优先选中最新，后续用户手动选择持续持久化。
|
||||
- 🎵 **背景音乐** - 试听 + 搜索选择 + 混音（当前前端固定混音系数，保持配音音量稳定）。
|
||||
- 🧩 **统一选择器交互** - 首页/发布页业务选择项统一 SelectPopover（桌面 Popover / 移动端 BottomSheet），支持自动上拉、已选定位与连续预览。
|
||||
- 🤖 **AI 辅助创作** - 内置 GLM-4.7-Flash，支持 B站/抖音链接文案提取、AI 智能改写（支持自定义提示词）、文案深度学习（博主话题分析+文案生成）、标题/标签自动生成、9 语言翻译。
|
||||
|
||||
### 平台化功能
|
||||
- 📱 **全自动发布** - 支持抖音/微信视频号/B站/小红书立即发布；扫码登录 + Cookie 持久化。
|
||||
- 🖥️ **发布管理预览** - 支持签名 URL / 相对路径作品预览，确保可直接播放。
|
||||
- 📸 **发布结果可视化** - 抖音/微信视频号/小红书发布成功后返回截图，发布页结果卡片可直接查看。
|
||||
- 🧹 **发布后工作区清理引导** - 全平台发布成功后弹出不可误关清理弹窗（失败可重试，达到阈值可暂不清理），仅清输入内容并保留用户偏好。
|
||||
- ⬇️ **一键下载直达** - 首页与发布成功弹窗下载统一走同源 `attachment` 接口，不再新开标签页播放视频。
|
||||
- 🛡️ **发布防误操作** - 发布进行中自动提示“请勿刷新或关闭网页”，并拦截刷新/关页二次确认。
|
||||
- 💳 **付费会员** - 支付宝电脑网站支付自动开通会员，到期自动停用并引导续费，管理员手动激活并存。
|
||||
- 🔐 **认证与隔离** - 基于 Supabase 的用户隔离，支持手机号注册/登录、密码管理。
|
||||
- 🛡️ **安全基线** - AI/Tools 接口强制登录鉴权、关键上传链路体积限制、生产环境默认密钥启动拦截。
|
||||
- 🛡️ **服务守护** - 内置 Watchdog 看门狗机制，自动监控并重启僵死服务，确保 7x24h 稳定运行。
|
||||
- 🖥️ **发布管理预览** - 支持签名 URL / 相对路径作品预览，确保可直接播放。
|
||||
- 📸 **发布结果可视化** - 抖音/微信视频号/小红书发布成功后返回截图，发布页结果卡片可直接查看。
|
||||
- 🧹 **发布后工作区清理引导** - 全平台发布成功后弹出不可误关清理弹窗（失败可重试，达到阈值可暂不清理），仅清输入内容并保留用户偏好。
|
||||
- ⬇️ **一键下载直达** - 首页与发布成功弹窗下载统一走同源 `attachment` 接口，不再新开标签页播放视频。
|
||||
- 🛡️ **发布防误操作** - 发布进行中自动提示“请勿刷新或关闭网页”，并拦截刷新/关页二次确认。
|
||||
- 💳 **付费会员** - 支付宝电脑网站支付自动开通会员，到期自动停用并引导续费，管理员手动激活并存。
|
||||
- 🔐 **认证与隔离** - 基于 Supabase 的用户隔离，支持手机号注册/登录、密码管理。
|
||||
- 🛡️ **安全基线** - AI/Tools 接口强制登录鉴权、关键上传链路体积限制、生产环境默认密钥启动拦截。
|
||||
- 🛡️ **服务守护** - 内置 Watchdog 看门狗机制，自动监控并重启僵死服务，确保 7x24h 稳定运行。
|
||||
- 🚀 **性能优化** - 编码流水线从 5-6 次有损编码精简至 3 次（prepare_segment → 模型输出 → Remotion）、compose 流复制免重编码、同分辨率跳过 scale、FFmpeg 超时保护、全局视频生成并发限制 (Semaphore(2))、Remotion 4 并发渲染、MuseTalk rawvideo 管道直编码（消除中间有损文件）、模型常驻服务、双 GPU 流水线并发、Redis 任务 TTL 自动清理、workflow 阻塞调用线程池化。
|
||||
|
||||
---
|
||||
@@ -64,9 +64,9 @@
|
||||
我们提供了详尽的开发与部署文档：
|
||||
|
||||
### 部署运维
|
||||
- **[部署手册 (DEPLOY_MANUAL.md)](Docs/DEPLOY_MANUAL.md)** - 👈 **部署请看这里**！包含完整的环境搭建步骤。
|
||||
- [多平台发布部署说明 (PUBLISH_DEPLOY.md)](Docs/PUBLISH_DEPLOY.md) - 抖音/微信视频号/B站/小红书登录与自动化发布专项文档。
|
||||
- [参考音频服务部署 (COSYVOICE3_DEPLOY.md)](Docs/COSYVOICE3_DEPLOY.md) - 声音克隆模型部署指南。
|
||||
- **[部署手册 (DEPLOY_MANUAL.md)](Docs/DEPLOY_MANUAL.md)** - 👈 **部署请看这里**！包含完整的环境搭建步骤。
|
||||
- [多平台发布部署说明 (PUBLISH_DEPLOY.md)](Docs/PUBLISH_DEPLOY.md) - 抖音/微信视频号/B站/小红书登录与自动化发布专项文档。
|
||||
- [参考音频服务部署 (COSYVOICE3_DEPLOY.md)](Docs/COSYVOICE3_DEPLOY.md) - 声音克隆模型部署指南。
|
||||
- [LatentSync 部署指南 (LATENTSYNC_DEPLOY.md)](Docs/LATENTSYNC_DEPLOY.md) - 唇形同步模型独立部署。
|
||||
- [MuseTalk 部署指南 (MUSETALK_DEPLOY.md)](Docs/MUSETALK_DEPLOY.md) - 长视频唇形同步模型部署。
|
||||
- [Supabase 部署指南 (SUPABASE_DEPLOY.md)](Docs/SUPABASE_DEPLOY.md) - Supabase 与认证系统配置。
|
||||
|
||||
@@ -1,15 +1,47 @@
|
||||
from fastapi import APIRouter, Depends, UploadFile, File, Form, HTTPException
|
||||
from typing import Optional
|
||||
from urllib.parse import urlparse
|
||||
import traceback
|
||||
from loguru import logger
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
from app.core.deps import get_current_user
|
||||
from app.core.response import success_response
|
||||
from app.modules.tools import service
|
||||
from app.services import creator_scraper
|
||||
from app.services.creator_scraper import ALLOWED_INPUT_DOMAINS
|
||||
from app.services.glm_service import glm_service
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
class AnalyzeCreatorRequest(BaseModel):
|
||||
url: str = Field(..., description="博主主页链接（仅支持抖音/B站 https 链接）")
|
||||
|
||||
@field_validator("url")
|
||||
@classmethod
|
||||
def validate_url_format(cls, value: str) -> str:
|
||||
candidate = value.strip()
|
||||
if len(candidate) > 500:
|
||||
raise ValueError("链接过长")
|
||||
|
||||
parsed = urlparse(candidate)
|
||||
if parsed.scheme != "https":
|
||||
raise ValueError("仅支持 https 链接")
|
||||
|
||||
hostname = (parsed.hostname or "").lower()
|
||||
if hostname not in ALLOWED_INPUT_DOMAINS:
|
||||
raise ValueError(f"不支持的域名: {hostname}，仅支持抖音和B站")
|
||||
|
||||
return candidate
|
||||
|
||||
|
||||
class GenerateTopicScriptRequest(BaseModel):
|
||||
analysis_id: str = Field(..., min_length=8, max_length=80, description="分析结果ID")
|
||||
topic: str = Field(..., min_length=2, max_length=30, description="选中的话题（2-30字）")
|
||||
word_count: int = Field(..., ge=80, le=1000, description="目标字数（80-1000）")
|
||||
|
||||
|
||||
@router.post("/extract-script")
|
||||
async def extract_script_tool(
|
||||
file: Optional[UploadFile] = File(None),
|
||||
@@ -33,3 +65,62 @@ async def extract_script_tool(
|
||||
if "Fresh cookies" in msg:
|
||||
raise HTTPException(500, "下载失败：目标平台开启了反爬验证，请过段时间重试或直接上传视频文件。")
|
||||
raise HTTPException(500, "文案提取失败，请稍后重试")
|
||||
|
||||
|
||||
@router.post("/analyze-creator")
|
||||
async def analyze_creator(
|
||||
req: AnalyzeCreatorRequest,
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""分析博主内容并返回热门话题"""
|
||||
try:
|
||||
user_id = str(current_user.get("id") or "").strip()
|
||||
if not user_id:
|
||||
raise HTTPException(401, "登录状态无效，请重新登录")
|
||||
|
||||
creator_result = await creator_scraper.scrape_creator_titles(req.url, user_id=user_id)
|
||||
titles = creator_result.get("titles") or []
|
||||
topics = await glm_service.analyze_topics(titles)
|
||||
|
||||
analysis_id = creator_scraper.cache_titles(titles, user_id)
|
||||
|
||||
return success_response({
|
||||
"platform": creator_result.get("platform", ""),
|
||||
"creator_name": creator_result.get("creator_name", ""),
|
||||
"topics": topics,
|
||||
"analysis_id": analysis_id,
|
||||
"fetched_count": creator_result.get("fetched_count", len(titles)),
|
||||
})
|
||||
except ValueError as e:
|
||||
raise HTTPException(400, str(e))
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"Analyze creator failed: {e}")
|
||||
logger.error(traceback.format_exc())
|
||||
raise HTTPException(500, "博主内容分析失败，请稍后重试")
|
||||
|
||||
|
||||
@router.post("/generate-topic-script")
|
||||
async def generate_topic_script(
|
||||
req: GenerateTopicScriptRequest,
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""根据话题生成文案"""
|
||||
try:
|
||||
user_id = str(current_user.get("id") or "").strip()
|
||||
if not user_id:
|
||||
raise HTTPException(401, "登录状态无效，请重新登录")
|
||||
|
||||
titles = creator_scraper.get_cached_titles(req.analysis_id, user_id)
|
||||
script = await glm_service.generate_script_from_topic(req.topic, req.word_count, titles)
|
||||
|
||||
return success_response({"script": script})
|
||||
except ValueError as e:
|
||||
raise HTTPException(400, str(e))
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"Generate topic script failed: {e}")
|
||||
logger.error(traceback.format_exc())
|
||||
raise HTTPException(500, "文案生成失败，请稍后重试")
|
||||
|
||||
@@ -8,7 +8,7 @@ import subprocess
|
||||
import traceback
|
||||
from pathlib import Path
|
||||
from typing import Optional, Any
|
||||
from urllib.parse import unquote
|
||||
from urllib.parse import unquote, parse_qs, urlparse
|
||||
|
||||
import httpx
|
||||
from loguru import logger
|
||||
@@ -212,10 +212,9 @@ async def _download_douyin_manual(url: str, temp_dir: Path, timestamp: int) -> O
|
||||
|
||||
logger.info(f"[douyin-fallback] Final URL: {final_url}")
|
||||
|
||||
video_id = None
|
||||
match = re.search(r'/video/(\d+)', final_url)
|
||||
if match:
|
||||
video_id = match.group(1)
|
||||
video_id = _extract_douyin_video_id(final_url)
|
||||
if not video_id:
|
||||
video_id = _extract_douyin_video_id(url)
|
||||
|
||||
if not video_id:
|
||||
logger.error("[douyin-fallback] Could not extract video_id")
|
||||
@@ -236,7 +235,8 @@ async def _download_douyin_manual(url: str, temp_dir: Path, timestamp: int) -> O
|
||||
"cbUrlProtocol": "https", "union": True,
|
||||
}
|
||||
)
|
||||
ttwid = ttwid_resp.cookies.get("ttwid", "")
|
||||
fresh_ttwid = ttwid_resp.cookies.get("ttwid")
|
||||
ttwid = str(fresh_ttwid) if fresh_ttwid else ""
|
||||
logger.info(f"[douyin-fallback] Got fresh ttwid (len={len(ttwid)})")
|
||||
except Exception as e:
|
||||
logger.warning(f"[douyin-fallback] Failed to get ttwid: {e}")
|
||||
@@ -296,6 +296,39 @@ async def _download_douyin_manual(url: str, temp_dir: Path, timestamp: int) -> O
|
||||
return None
|
||||
|
||||
|
||||
def _extract_douyin_video_id(candidate_url: str) -> Optional[str]:
|
||||
"""从抖音 URL 中提取视频 ID，兼容 video/share/video/modal_id/vid 等形态"""
|
||||
if not candidate_url:
|
||||
return None
|
||||
|
||||
decoded_url = unquote(candidate_url)
|
||||
parsed = urlparse(decoded_url)
|
||||
|
||||
for source in (decoded_url, parsed.path):
|
||||
for pattern in (r"/video/(\d+)", r"/share/video/(\d+)"):
|
||||
match = re.search(pattern, source)
|
||||
if match:
|
||||
return match.group(1)
|
||||
|
||||
id_keys = ("modal_id", "vid", "video_id", "aweme_id", "item_id")
|
||||
for pairs in (parse_qs(parsed.query), parse_qs(parsed.fragment)):
|
||||
for key in id_keys:
|
||||
values = pairs.get(key, [])
|
||||
for value in values:
|
||||
match = re.search(r"(\d+)", value)
|
||||
if match:
|
||||
return match.group(1)
|
||||
|
||||
inline_match = re.search(
|
||||
r"(?:[?&#](?:modal_id|vid|video_id|aweme_id|item_id)=)(\d+)",
|
||||
decoded_url,
|
||||
)
|
||||
if inline_match:
|
||||
return inline_match.group(1)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
async def _download_bilibili_manual(url: str, temp_dir: Path, timestamp: int) -> Optional[Path]:
|
||||
"""手动下载 Bilibili 视频 (Playwright Fallback)"""
|
||||
from playwright.async_api import async_playwright
|
||||
|
||||
@@ -188,16 +188,16 @@ async def _process_video_generation_inner(task_id: str, req: GenerateRequest, us
|
||||
try:
|
||||
start_time = time.time()
|
||||
|
||||
# ── 确定素材列表 ──
|
||||
# ── 确定素材列表（优先信任 req.material_paths 去重列表）──
|
||||
material_paths: List[str] = []
|
||||
if req.custom_assignments and len(req.custom_assignments) > 1:
|
||||
material_paths = [a.material_path for a in req.custom_assignments if a.material_path]
|
||||
elif req.material_paths and len(req.material_paths) > 1:
|
||||
if req.material_paths and len(req.material_paths) >= 1:
|
||||
material_paths = req.material_paths
|
||||
else:
|
||||
material_paths = [req.material_path]
|
||||
|
||||
is_multi = len(material_paths) > 1
|
||||
is_multi = len(material_paths) > 1 or (
|
||||
req.custom_assignments is not None and len(req.custom_assignments) > 1
|
||||
)
|
||||
target_resolution = (1080, 1920) if req.output_aspect_ratio == "9:16" else (1920, 1080)
|
||||
|
||||
logger.info(
|
||||
@@ -341,8 +341,18 @@ async def _process_video_generation_inner(task_id: str, req: GenerateRequest, us
|
||||
# ══════════════════════════════════════
|
||||
_update_task(task_id, progress=12, message="正在分配素材...")
|
||||
|
||||
if req.custom_assignments and len(req.custom_assignments) == len(material_paths):
|
||||
# 用户自定义分配，跳过 Whisper 均分
|
||||
if req.custom_assignments and len(req.custom_assignments) >= 1:
|
||||
# 硬上限校验
|
||||
if len(req.custom_assignments) > 50:
|
||||
raise ValueError(f"custom_assignments 数量超限: {len(req.custom_assignments)}")
|
||||
# 校验所有 assignment 的 material_path 都在前端声明的 material_paths 中
|
||||
known_paths = set(material_paths)
|
||||
unknown = [a.material_path for a in req.custom_assignments if a.material_path not in known_paths]
|
||||
if unknown:
|
||||
logger.warning(f"[MultiMat] custom_assignments 包含未知素材路径: {unknown[:3]}，终止生成")
|
||||
raise ValueError(f"素材路径校验失败: 包含 {len(unknown)} 个未知路径")
|
||||
|
||||
# 用户自定义分配（多镜头模式：主素材可重复出现）
|
||||
assignments = [
|
||||
{
|
||||
"material_path": a.material_path,
|
||||
@@ -373,20 +383,13 @@ async def _process_video_generation_inner(task_id: str, req: GenerateRequest, us
|
||||
captions_path = None
|
||||
else:
|
||||
captions_path = None
|
||||
elif req.custom_assignments:
|
||||
logger.warning(
|
||||
f"[MultiMat] custom_assignments 数量({len(req.custom_assignments)})"
|
||||
f" 与素材数量({len(material_paths)})不一致，回退自动分配"
|
||||
)
|
||||
|
||||
assignments, captions_path = await _whisper_and_split()
|
||||
|
||||
else:
|
||||
assignments, captions_path = await _whisper_and_split()
|
||||
|
||||
# 扩展段覆盖完整音频范围：首段从0开始，末段到音频结尾
|
||||
# 扩展段覆盖完整音频范围（仅自动均分时执行，自定义分配已精确计算）
|
||||
audio_duration = await _run_blocking(video._get_duration, str(audio_path))
|
||||
if assignments and audio_duration > 0:
|
||||
if not req.custom_assignments and assignments and audio_duration > 0:
|
||||
assignments[0]["start"] = 0.0
|
||||
assignments[-1]["end"] = audio_duration
|
||||
|
||||
@@ -398,65 +401,73 @@ async def _process_video_generation_inner(task_id: str, req: GenerateRequest, us
|
||||
|
||||
lipsync_start = time.time()
|
||||
|
||||
# ── 第一步：并行下载所有素材并检测分辨率 ──
|
||||
material_locals: List[Path] = []
|
||||
resolutions = []
|
||||
# 并发限流（每个任务独立 Semaphore，峰值 2×4=8 个 ffmpeg 进程）
|
||||
_segment_sem = asyncio.Semaphore(4)
|
||||
|
||||
async def _download_and_normalize(i: int, assignment: dict):
|
||||
"""下载单个素材并归一化方向"""
|
||||
material_local = temp_dir / f"{task_id}_material_{i}.mp4"
|
||||
temp_files.append(material_local)
|
||||
await _download_material(assignment["material_path"], material_local)
|
||||
# ── 第一步：去重下载所有素材并检测分辨率 ──
|
||||
unique_paths = list(dict.fromkeys(a["material_path"] for a in assignments))
|
||||
path_to_local: dict = {} # material_path → 本地文件
|
||||
path_to_res: dict = {} # material_path → 分辨率
|
||||
|
||||
normalized_material = temp_dir / f"{task_id}_material_{i}_norm.mp4"
|
||||
normalized_result = await _run_blocking(
|
||||
video.normalize_orientation,
|
||||
str(material_local),
|
||||
str(normalized_material),
|
||||
)
|
||||
if normalized_result != str(material_local):
|
||||
temp_files.append(normalized_material)
|
||||
material_local = normalized_material
|
||||
async def _download_unique(mat_path: str, idx: int):
|
||||
"""去重下载单个素材并归一化方向"""
|
||||
async with _segment_sem:
|
||||
material_local = temp_dir / f"{task_id}_material_{idx}.mp4"
|
||||
temp_files.append(material_local)
|
||||
await _download_material(mat_path, material_local)
|
||||
|
||||
res = video.get_resolution(str(material_local))
|
||||
return material_local, res
|
||||
normalized_material = temp_dir / f"{task_id}_material_{idx}_norm.mp4"
|
||||
normalized_result = await _run_blocking(
|
||||
video.normalize_orientation,
|
||||
str(material_local),
|
||||
str(normalized_material),
|
||||
)
|
||||
if normalized_result != str(material_local):
|
||||
temp_files.append(normalized_material)
|
||||
material_local = normalized_material
|
||||
|
||||
download_tasks = [
|
||||
_download_and_normalize(i, assignment)
|
||||
for i, assignment in enumerate(assignments)
|
||||
]
|
||||
download_results = await asyncio.gather(*download_tasks)
|
||||
for local, res in download_results:
|
||||
material_locals.append(local)
|
||||
resolutions.append(res)
|
||||
res = video.get_resolution(str(material_local))
|
||||
return mat_path, material_local, res
|
||||
|
||||
download_results = await asyncio.gather(*[
|
||||
_download_unique(p, i) for i, p in enumerate(unique_paths)
|
||||
])
|
||||
for mat_path, local, res in download_results:
|
||||
path_to_local[mat_path] = local
|
||||
path_to_res[mat_path] = res
|
||||
|
||||
logger.info(f"[MultiMat] 去重下载 {len(unique_paths)} 个素材（共 {num_segments} 个段）")
|
||||
|
||||
# 按用户选择的画面比例统一分辨率
|
||||
base_res = target_resolution
|
||||
need_scale = any(r != base_res for r in resolutions)
|
||||
need_scale = any(r != base_res for r in path_to_res.values())
|
||||
if need_scale:
|
||||
logger.info(f"[MultiMat] 素材分辨率不一致，统一到 {base_res[0]}x{base_res[1]}")
|
||||
|
||||
# ── 第二步：并行裁剪每段素材到对应时长 ──
|
||||
# ── 第二步：并行裁剪每段素材到对应时长（通过映射找到已下载文件）──
|
||||
prepared_segments: List[Optional[Path]] = [None] * num_segments
|
||||
|
||||
async def _prepare_one_segment(i: int, assignment: dict):
|
||||
"""将单个素材裁剪/循环到对应时长"""
|
||||
seg_dur = assignment["end"] - assignment["start"]
|
||||
prepared_path = temp_dir / f"{task_id}_prepared_{i}.mp4"
|
||||
temp_files.append(prepared_path)
|
||||
prepare_target_res = None if resolutions[i] == base_res else base_res
|
||||
async with _segment_sem:
|
||||
seg_dur = assignment["end"] - assignment["start"]
|
||||
prepared_path = temp_dir / f"{task_id}_prepared_{i}.mp4"
|
||||
temp_files.append(prepared_path)
|
||||
mat_local = path_to_local[assignment["material_path"]]
|
||||
mat_res = path_to_res[assignment["material_path"]]
|
||||
prepare_target_res = None if mat_res == base_res else base_res
|
||||
|
||||
await _run_blocking(
|
||||
video.prepare_segment,
|
||||
str(material_locals[i]),
|
||||
seg_dur,
|
||||
str(prepared_path),
|
||||
prepare_target_res,
|
||||
assignment.get("source_start", 0.0),
|
||||
assignment.get("source_end"),
|
||||
25,
|
||||
)
|
||||
return i, prepared_path
|
||||
await _run_blocking(
|
||||
video.prepare_segment,
|
||||
str(mat_local),
|
||||
seg_dur,
|
||||
str(prepared_path),
|
||||
prepare_target_res,
|
||||
assignment.get("source_start", 0.0),
|
||||
assignment.get("source_end"),
|
||||
25,
|
||||
)
|
||||
return i, prepared_path
|
||||
|
||||
_update_task(
|
||||
task_id,
|
||||
|
||||
1301
backend/app/services/creator_scraper.py
Normal file
1301
backend/app/services/creator_scraper.py
Normal file
File diff suppressed because it is too large
Load Diff
@@ -3,8 +3,10 @@ GLM AI 服务
|
||||
使用智谱 GLM 生成标题和标签
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import re
|
||||
from typing import Any, Optional, cast
|
||||
from loguru import logger
|
||||
from zai import ZhipuAiClient
|
||||
|
||||
@@ -25,6 +27,48 @@ class GLMService:
|
||||
self.client = ZhipuAiClient(api_key=settings.GLM_API_KEY)
|
||||
return self.client
|
||||
|
||||
async def _call_glm(
|
||||
self,
|
||||
*,
|
||||
prompt: str,
|
||||
max_tokens: int,
|
||||
temperature: float,
|
||||
action: str,
|
||||
timeout_seconds: float = 85.0,
|
||||
) -> str:
|
||||
"""统一 GLM 调用入口，避免重复调用代码"""
|
||||
client = self._get_client()
|
||||
logger.info(
|
||||
f"{action} | model={settings.GLM_MODEL} | max_tokens={max_tokens} | temperature={temperature}"
|
||||
)
|
||||
|
||||
try:
|
||||
response = await asyncio.wait_for(
|
||||
asyncio.to_thread(
|
||||
client.chat.completions.create,
|
||||
model=settings.GLM_MODEL,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
thinking={"type": "disabled"},
|
||||
max_tokens=max_tokens,
|
||||
temperature=temperature,
|
||||
),
|
||||
timeout=timeout_seconds,
|
||||
)
|
||||
except asyncio.TimeoutError as exc:
|
||||
raise Exception("GLM 请求超时，请稍后重试") from exc
|
||||
|
||||
completion = cast(Any, response)
|
||||
choices = getattr(completion, "choices", None)
|
||||
if not choices:
|
||||
raise Exception("AI 返回内容为空")
|
||||
|
||||
message = getattr(choices[0], "message", None)
|
||||
content = getattr(message, "content", "")
|
||||
text = content.strip() if isinstance(content, str) else str(content or "").strip()
|
||||
if not text:
|
||||
raise Exception("AI 返回内容为空")
|
||||
return text
|
||||
|
||||
async def generate_title_tags(self, text: str) -> dict:
|
||||
"""
|
||||
根据口播文案生成标题和标签
|
||||
@@ -50,22 +94,13 @@ class GLMService:
|
||||
{{"title": "标题", "secondary_title": "副标题", "tags": ["标签1", "标签2", "标签3"]}}"""
|
||||
|
||||
try:
|
||||
client = self._get_client()
|
||||
logger.info(f"Calling GLM API with model: {settings.GLM_MODEL}")
|
||||
|
||||
# 使用 asyncio.to_thread 包装同步 SDK 调用，避免阻塞事件循环
|
||||
import asyncio
|
||||
response = await asyncio.to_thread(
|
||||
client.chat.completions.create,
|
||||
model=settings.GLM_MODEL,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
thinking={"type": "disabled"}, # 禁用思考模式，加快响应
|
||||
content = await self._call_glm(
|
||||
prompt=prompt,
|
||||
max_tokens=500,
|
||||
temperature=0.7
|
||||
temperature=0.7,
|
||||
action="生成标题与标签",
|
||||
timeout_seconds=75.0,
|
||||
)
|
||||
|
||||
# 提取生成的内容
|
||||
content = response.choices[0].message.content
|
||||
logger.info(f"GLM response (model: {settings.GLM_MODEL}): {content}")
|
||||
|
||||
# 解析 JSON
|
||||
@@ -76,7 +111,7 @@ class GLMService:
|
||||
logger.error(f"GLM service error: {e}")
|
||||
raise Exception(f"AI 生成失败: {str(e)}")
|
||||
|
||||
async def rewrite_script(self, text: str, custom_prompt: str = None) -> str:
|
||||
async def rewrite_script(self, text: str, custom_prompt: Optional[str] = None) -> str:
|
||||
"""
|
||||
AI 改写文案
|
||||
|
||||
@@ -105,28 +140,126 @@ class GLMService:
|
||||
4. 不要返回多余的解释，只返回改写后的正文"""
|
||||
|
||||
try:
|
||||
client = self._get_client()
|
||||
logger.info(f"Using GLM to rewrite script")
|
||||
|
||||
# 使用 asyncio.to_thread 包装同步 SDK 调用，避免阻塞事件循环
|
||||
import asyncio
|
||||
response = await asyncio.to_thread(
|
||||
client.chat.completions.create,
|
||||
model=settings.GLM_MODEL,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
thinking={"type": "disabled"},
|
||||
content = await self._call_glm(
|
||||
prompt=prompt,
|
||||
max_tokens=2000,
|
||||
temperature=0.8
|
||||
temperature=0.8,
|
||||
action="改写文案",
|
||||
timeout_seconds=85.0,
|
||||
)
|
||||
|
||||
content = response.choices[0].message.content
|
||||
logger.info("GLM rewrite completed")
|
||||
return content.strip()
|
||||
return content
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"GLM rewrite error: {e}")
|
||||
raise Exception(f"AI 改写失败: {str(e)}")
|
||||
|
||||
async def analyze_topics(self, titles: list[str]) -> list[str]:
|
||||
"""
|
||||
分析视频标题列表并归纳热门话题（最多 10 个）
|
||||
"""
|
||||
cleaned_titles = [str(title).strip() for title in titles if str(title).strip()]
|
||||
if not cleaned_titles:
|
||||
raise Exception("标题列表为空")
|
||||
|
||||
limited_titles = cleaned_titles[:50]
|
||||
titles_text = "\n".join(f"{idx + 1}. {title}" for idx, title in enumerate(limited_titles))
|
||||
|
||||
prompt = f"""以下是某短视频博主最近发布的视频标题列表：
|
||||
|
||||
{titles_text}
|
||||
|
||||
请分析这些标题，归纳总结出该博主内容中最热门的话题方向。
|
||||
|
||||
要求：
|
||||
1. 提取不超过10个话题方向
|
||||
2. 每个话题用简短短语描述（建议 5-15 字）
|
||||
3. 按热门程度排序（出现频率高的在前）
|
||||
4. 只返回话题列表，每行一个，不要编号、解释或多余内容"""
|
||||
|
||||
try:
|
||||
content = await self._call_glm(
|
||||
prompt=prompt,
|
||||
max_tokens=500,
|
||||
temperature=0.5,
|
||||
action="分析博主话题",
|
||||
timeout_seconds=85.0,
|
||||
)
|
||||
topics = self._parse_topic_lines(content)
|
||||
if not topics:
|
||||
raise Exception("未识别到有效话题")
|
||||
|
||||
logger.info(f"GLM topic analysis completed: {len(topics)} topics")
|
||||
return topics[:10]
|
||||
except Exception as e:
|
||||
logger.error(f"GLM topic analysis error: {e}")
|
||||
raise Exception(f"话题分析失败: {str(e)}")
|
||||
|
||||
async def generate_script_from_topic(self, topic: str, word_count: int, titles: list[str]) -> str:
|
||||
"""
|
||||
根据选中话题与博主标题风格生成文案
|
||||
"""
|
||||
topic_value = str(topic or "").strip()
|
||||
if not topic_value:
|
||||
raise Exception("话题不能为空")
|
||||
|
||||
cleaned_titles = [str(title).strip() for title in titles if str(title).strip()]
|
||||
if not cleaned_titles:
|
||||
raise Exception("参考标题为空")
|
||||
|
||||
word_count_value = max(80, min(int(word_count), 1000))
|
||||
sample_titles = "\n".join(f"{idx + 1}. {title}" for idx, title in enumerate(cleaned_titles[:10]))
|
||||
|
||||
prompt = f"""请围绕「{topic_value}」这个话题，生成一段短视频口播文案。
|
||||
|
||||
参考该博主的标题风格：
|
||||
{sample_titles}
|
||||
|
||||
要求：
|
||||
1. 文案字数约 {word_count_value} 字
|
||||
2. 适合短视频口播，语气自然、有吸引力
|
||||
3. 开头要有钩子吸引观众
|
||||
4. 只返回文案正文，不要标题和其他说明"""
|
||||
|
||||
try:
|
||||
content = await self._call_glm(
|
||||
prompt=prompt,
|
||||
max_tokens=min(word_count_value * 3, 4000),
|
||||
temperature=0.8,
|
||||
action=f"按话题生成文案(topic={topic_value})",
|
||||
timeout_seconds=88.0,
|
||||
)
|
||||
|
||||
logger.info("GLM topic script generation completed")
|
||||
return content
|
||||
except Exception as e:
|
||||
logger.error(f"GLM topic script generation error: {e}")
|
||||
raise Exception(f"文案生成失败: {str(e)}")
|
||||
|
||||
def _parse_topic_lines(self, content: str) -> list[str]:
|
||||
lines = [line.strip() for line in str(content or "").splitlines()]
|
||||
topics: list[str] = []
|
||||
seen: set[str] = set()
|
||||
|
||||
for line in lines:
|
||||
if not line:
|
||||
continue
|
||||
|
||||
cleaned = re.sub(r"^\s*(?:[-*•]+|\d+[.)、\s]+)", "", line).strip()
|
||||
cleaned = cleaned.strip('"“”')
|
||||
if not cleaned:
|
||||
continue
|
||||
|
||||
if cleaned in seen:
|
||||
continue
|
||||
seen.add(cleaned)
|
||||
topics.append(cleaned)
|
||||
|
||||
if len(topics) >= 10:
|
||||
break
|
||||
|
||||
return topics
|
||||
|
||||
|
||||
|
||||
async def translate_text(self, text: str, target_lang: str) -> str:
|
||||
@@ -151,22 +284,15 @@ class GLMService:
|
||||
3. 翻译要自然流畅，符合目标语言的表达习惯"""
|
||||
|
||||
try:
|
||||
client = self._get_client()
|
||||
logger.info(f"Using GLM to translate text to {target_lang}")
|
||||
|
||||
import asyncio
|
||||
response = await asyncio.to_thread(
|
||||
client.chat.completions.create,
|
||||
model=settings.GLM_MODEL,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
thinking={"type": "disabled"},
|
||||
content = await self._call_glm(
|
||||
prompt=prompt,
|
||||
max_tokens=2000,
|
||||
temperature=0.3
|
||||
temperature=0.3,
|
||||
action=f"翻译文案(target={target_lang})",
|
||||
timeout_seconds=75.0,
|
||||
)
|
||||
|
||||
content = response.choices[0].message.content
|
||||
logger.info("GLM translation completed")
|
||||
return content.strip()
|
||||
return content
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"GLM translate error: {e}")
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { useEffect, useMemo, useRef, useState } from "react";
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
import api from "@/shared/api/axios";
|
||||
import {
|
||||
buildTextShadow,
|
||||
@@ -256,6 +256,14 @@ export const useHomeController = () => {
|
||||
const payload = unwrap(res);
|
||||
if (selectedMaterials.includes(materialId) && payload?.id) {
|
||||
setSelectedMaterials((prev) => prev.map((x) => (x === materialId ? payload.id : x)));
|
||||
// Sync inserts: update materialId and name when rename changes the ID
|
||||
if (payload.id !== materialId) {
|
||||
setInserts((prev) => prev.map((ins) =>
|
||||
ins.materialId === materialId
|
||||
? { ...ins, materialId: payload.id, materialName: editMaterialName.trim() }
|
||||
: ins
|
||||
));
|
||||
}
|
||||
}
|
||||
setEditingMaterialId(null);
|
||||
setEditMaterialName("");
|
||||
@@ -287,6 +295,9 @@ export const useHomeController = () => {
|
||||
// 文案提取模态框
|
||||
const [extractModalOpen, setExtractModalOpen] = useState(false);
|
||||
|
||||
// 文案深度学习模态框
|
||||
const [learningModalOpen, setLearningModalOpen] = useState(false);
|
||||
|
||||
// AI 改写模态框
|
||||
const [rewriteModalOpen, setRewriteModalOpen] = useState(false);
|
||||
|
||||
@@ -307,6 +318,7 @@ export const useHomeController = () => {
|
||||
setUploadError,
|
||||
fetchMaterials,
|
||||
toggleMaterial,
|
||||
reorderMaterials,
|
||||
deleteMaterial,
|
||||
handleUpload,
|
||||
} = useMaterials({
|
||||
@@ -394,9 +406,17 @@ export const useHomeController = () => {
|
||||
});
|
||||
|
||||
const {
|
||||
segments: timelineSegments,
|
||||
reorderSegments,
|
||||
setSourceRange,
|
||||
inserts,
|
||||
setInserts,
|
||||
primaryMaterial: timelinePrimaryMaterial,
|
||||
primarySourceStart,
|
||||
primarySourceEnd,
|
||||
addInsert,
|
||||
removeInsert,
|
||||
moveInsert,
|
||||
resizeInsert,
|
||||
setInsertSourceRange,
|
||||
setPrimarySourceRange,
|
||||
toCustomAssignments,
|
||||
} = useTimelineEditor({
|
||||
audioDuration: selectedAudio?.duration_sec ?? 0,
|
||||
@@ -405,16 +425,15 @@ export const useHomeController = () => {
|
||||
storageKey,
|
||||
});
|
||||
|
||||
// 时间轴第一段素材的视频 URL（用于帧截取预览）
|
||||
// 主素材的视频 URL（用于帧截取预览）
|
||||
// 使用后端代理 URL（同源）避免 CORS canvas taint
|
||||
const firstTimelineMaterialUrl = useMemo(() => {
|
||||
const firstSeg = timelineSegments[0];
|
||||
const matId = firstSeg?.materialId ?? selectedMaterials[0];
|
||||
const matId = selectedMaterials[0];
|
||||
if (!matId) return null;
|
||||
const mat = materials.find((m) => m.id === matId);
|
||||
if (!mat) return null;
|
||||
return `/api/materials/stream/${mat.id}`;
|
||||
}, [materials, timelineSegments, selectedMaterials]);
|
||||
}, [materials, selectedMaterials]);
|
||||
|
||||
const materialPosterUrl = useVideoFrameCapture(showStylePreview ? firstTimelineMaterialUrl : null);
|
||||
|
||||
@@ -952,57 +971,36 @@ export const useHomeController = () => {
|
||||
output_aspect_ratio: outputAspectRatio,
|
||||
};
|
||||
|
||||
// 多素材
|
||||
// 多素材（多镜头模式）
|
||||
if (selectedMaterials.length > 1) {
|
||||
const timelineOrderedIds = timelineSegments
|
||||
.map((seg) => seg.materialId)
|
||||
.filter((id, index, arr) => arr.indexOf(id) === index);
|
||||
const orderedMaterialIds = [
|
||||
...timelineOrderedIds.filter((id) => selectedMaterials.includes(id)),
|
||||
...selectedMaterials.filter((id) => !timelineOrderedIds.includes(id)),
|
||||
];
|
||||
|
||||
const materialPaths = orderedMaterialIds
|
||||
.map((id) => materials.find((x) => x.id === id)?.path)
|
||||
.filter((path): path is string => !!path);
|
||||
|
||||
if (materialPaths.length === 0) {
|
||||
toast.error("多素材解析失败，请刷新素材后重试");
|
||||
return;
|
||||
}
|
||||
|
||||
payload.material_paths = materialPaths;
|
||||
payload.material_path = materialPaths[0];
|
||||
|
||||
// 发送自定义时间轴分配
|
||||
const assignments = toCustomAssignments();
|
||||
if (assignments.length > 0) {
|
||||
const assignmentPaths = assignments
|
||||
.map((a) => a.material_path)
|
||||
.filter((path): path is string => !!path);
|
||||
|
||||
if (assignmentPaths.length === assignments.length) {
|
||||
// 以时间轴可见段为准：超出时间轴的素材不会参与本次生成
|
||||
payload.material_paths = assignmentPaths;
|
||||
payload.material_path = assignmentPaths[0];
|
||||
// 前端预估段数校验（与后端硬上限 50 对齐）
|
||||
if (assignments.length > 50) {
|
||||
toast.error(`时间轴段数过多（${assignments.length}），请减少插入或使用更长的主素材`);
|
||||
return;
|
||||
}
|
||||
// 主素材路径（始终来自 selectedMaterials[0]）
|
||||
const primaryPath = firstMaterialObj.path;
|
||||
// 去重素材路径列表，主素材保证在首位
|
||||
const otherPaths = [...new Set(
|
||||
assignments.map((a) => a.material_path).filter((p) => p !== primaryPath)
|
||||
)];
|
||||
payload.material_path = primaryPath;
|
||||
payload.material_paths = [primaryPath, ...otherPaths];
|
||||
payload.custom_assignments = assignments;
|
||||
} else {
|
||||
console.warn(
|
||||
"[Timeline] custom_assignments 为空，回退后端自动分配",
|
||||
{ materials: materialPaths.length }
|
||||
);
|
||||
// 无插入且主素材无裁剪：退化为单素材
|
||||
payload.material_path = firstMaterialObj.path;
|
||||
}
|
||||
}
|
||||
|
||||
// 单素材 + 截取范围
|
||||
const singleSeg = timelineSegments[0];
|
||||
if (
|
||||
selectedMaterials.length === 1
|
||||
&& singleSeg
|
||||
&& (singleSeg.sourceStart > 0 || singleSeg.sourceEnd > 0)
|
||||
) {
|
||||
payload.custom_assignments = toCustomAssignments();
|
||||
if (selectedMaterials.length === 1) {
|
||||
const assignments = toCustomAssignments();
|
||||
if (assignments.length > 0) {
|
||||
payload.custom_assignments = assignments;
|
||||
}
|
||||
}
|
||||
|
||||
if (selectedSubtitleStyleId) {
|
||||
@@ -1094,6 +1092,21 @@ export const useHomeController = () => {
|
||||
videoItemRefs.current[id] = el;
|
||||
};
|
||||
|
||||
// 设为主素材：将目标素材移到 selectedMaterials[0]
|
||||
const handleSetPrimary = useCallback((materialId: string) => {
|
||||
setSelectedMaterials((prev) => {
|
||||
const filtered = prev.filter((id) => id !== materialId);
|
||||
return [materialId, ...filtered];
|
||||
});
|
||||
}, [setSelectedMaterials]);
|
||||
|
||||
// 多镜头：插入候选素材（selectedMaterials[1:]）
|
||||
const insertCandidates = useMemo(() => {
|
||||
return selectedMaterials.slice(1)
|
||||
.map((id) => materials.find((m) => m.id === id))
|
||||
.filter((m): m is Material => !!m);
|
||||
}, [selectedMaterials, materials]);
|
||||
|
||||
return {
|
||||
apiBase,
|
||||
registerMaterialRef,
|
||||
@@ -1123,6 +1136,8 @@ export const useHomeController = () => {
|
||||
setText,
|
||||
extractModalOpen,
|
||||
setExtractModalOpen,
|
||||
learningModalOpen,
|
||||
setLearningModalOpen,
|
||||
rewriteModalOpen,
|
||||
setRewriteModalOpen,
|
||||
handleGenerateMeta,
|
||||
@@ -1246,9 +1261,20 @@ export const useHomeController = () => {
|
||||
setSpeed,
|
||||
emotion,
|
||||
setEmotion,
|
||||
timelineSegments,
|
||||
reorderSegments,
|
||||
setSourceRange,
|
||||
// Multi-camera timeline
|
||||
inserts,
|
||||
timelinePrimaryMaterial,
|
||||
primarySourceStart,
|
||||
primarySourceEnd,
|
||||
insertCandidates,
|
||||
addInsert,
|
||||
removeInsert,
|
||||
moveInsert,
|
||||
resizeInsert,
|
||||
setInsertSourceRange,
|
||||
setPrimarySourceRange,
|
||||
handleSetPrimary,
|
||||
reorderMaterials,
|
||||
clipTrimmerOpen,
|
||||
setClipTrimmerOpen,
|
||||
clipTrimmerSegmentId,
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
import { useCallback, useEffect, useRef, useState } from "react";
|
||||
import type { Material } from "@/shared/types/material";
|
||||
import type { InsertSegment } from "@/shared/types/timeline";
|
||||
|
||||
// Re-export for downstream consumers (ClipTrimmer, etc.)
|
||||
export type { InsertSegment };
|
||||
|
||||
export interface TimelineSegment {
|
||||
id: string;
|
||||
@@ -12,18 +16,23 @@ export interface TimelineSegment {
|
||||
color: string;
|
||||
}
|
||||
|
||||
export interface CustomAssignment {
|
||||
material_path: string;
|
||||
start: number;
|
||||
end: number;
|
||||
source_start: number;
|
||||
source_end?: number;
|
||||
}
|
||||
export interface CustomAssignment {
|
||||
material_path: string;
|
||||
start: number;
|
||||
end: number;
|
||||
source_start: number;
|
||||
source_end?: number;
|
||||
}
|
||||
|
||||
const COLORS = ["#8b5cf6", "#ec4899", "#06b6d4", "#f59e0b", "#10b981", "#f97316"];
|
||||
const MAX_INSERTS = 10;
|
||||
const DEFAULT_INSERT_DURATION = 3;
|
||||
const MIN_GAP = 0.5;
|
||||
|
||||
export type AddInsertResult = "ok" | "limit" | "no_space";
|
||||
|
||||
/** Serializable subset for localStorage */
|
||||
interface SegmentSnapshot {
|
||||
interface InsertSnapshot {
|
||||
materialId: string;
|
||||
start: number;
|
||||
end: number;
|
||||
@@ -31,56 +40,11 @@ interface SegmentSnapshot {
|
||||
sourceEnd: number;
|
||||
}
|
||||
|
||||
/** Get effective duration of a segment (clipped range or full material duration) */
|
||||
function getEffectiveDuration(
|
||||
seg: { sourceStart: number; sourceEnd: number; materialId: string },
|
||||
mats: Material[]
|
||||
): number {
|
||||
const mat = mats.find((m) => m.id === seg.materialId);
|
||||
const matDur = mat?.duration_sec ?? 0;
|
||||
if (seg.sourceEnd > seg.sourceStart) return seg.sourceEnd - seg.sourceStart;
|
||||
if (seg.sourceStart > 0) return Math.max(matDur - seg.sourceStart, 0);
|
||||
return matDur;
|
||||
}
|
||||
|
||||
/**
|
||||
* Recalculate segment start/end positions based on effective durations.
|
||||
* - Segments placed sequentially by effective duration
|
||||
* - Segments exceeding audioDuration keep their positions (overflow, start >= duration)
|
||||
* - Last visible segment is capped/extended to exactly audioDuration (loop fill)
|
||||
*/
|
||||
function recalcPositions(
|
||||
segs: TimelineSegment[],
|
||||
mats: Material[],
|
||||
duration: number
|
||||
): TimelineSegment[] {
|
||||
if (segs.length === 0 || duration <= 0) return segs;
|
||||
|
||||
const fallbackDur = duration / segs.length;
|
||||
let cursor = 0;
|
||||
const result = segs.map((seg) => {
|
||||
const effDur = getEffectiveDuration(seg, mats);
|
||||
const dur = effDur > 0 ? effDur : fallbackDur;
|
||||
const newSeg = { ...seg, start: cursor, end: cursor + dur };
|
||||
cursor += dur;
|
||||
return newSeg;
|
||||
});
|
||||
|
||||
// Find last segment that starts before audioDuration
|
||||
let lastVisibleIdx = -1;
|
||||
for (let i = result.length - 1; i >= 0; i--) {
|
||||
if (result[i].start < duration) {
|
||||
lastVisibleIdx = i;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// Cap/extend last visible segment to exactly audioDuration
|
||||
if (lastVisibleIdx >= 0) {
|
||||
result[lastVisibleIdx] = { ...result[lastVisibleIdx], end: duration };
|
||||
}
|
||||
|
||||
return result;
|
||||
interface MultiCamCache {
|
||||
key: string;
|
||||
inserts: InsertSnapshot[];
|
||||
primarySourceStart: number;
|
||||
primarySourceEnd: number;
|
||||
}
|
||||
|
||||
interface UseTimelineEditorOptions {
|
||||
@@ -96,34 +60,40 @@ export const useTimelineEditor = ({
|
||||
selectedMaterials,
|
||||
storageKey,
|
||||
}: UseTimelineEditorOptions) => {
|
||||
const [segments, setSegments] = useState<TimelineSegment[]>([]);
|
||||
const [inserts, setInserts] = useState<InsertSegment[]>([]);
|
||||
const [primarySourceStart, setPrimarySourceStart] = useState(0);
|
||||
const [primarySourceEnd, setPrimarySourceEnd] = useState(0);
|
||||
const prevKey = useRef("");
|
||||
const restoredRef = useRef(false);
|
||||
const [prevPrimaryId, setPrevPrimaryId] = useState(selectedMaterials[0]);
|
||||
|
||||
// Refs for stable callbacks (avoid recreating on every materials/duration change)
|
||||
const materialsRef = useRef(materials);
|
||||
const audioDurationRef = useRef(audioDuration);
|
||||
|
||||
useEffect(() => {
|
||||
materialsRef.current = materials;
|
||||
}, [materials]);
|
||||
|
||||
useEffect(() => {
|
||||
audioDurationRef.current = audioDuration;
|
||||
}, [audioDuration]);
|
||||
// Refs for stable callbacks
|
||||
const materialsRef = useRef(materials);
|
||||
const audioDurationRef = useRef(audioDuration);
|
||||
const selectedMaterialsRef = useRef(selectedMaterials);
|
||||
|
||||
// Build a durationsKey so segments re-init when material durations become available
|
||||
const durationsKey = selectedMaterials
|
||||
.map((id) => materials.find((m) => m.id === id)?.duration_sec ?? 0)
|
||||
.join(",");
|
||||
useEffect(() => { materialsRef.current = materials; }, [materials]);
|
||||
useEffect(() => { audioDurationRef.current = audioDuration; }, [audioDuration]);
|
||||
useEffect(() => { selectedMaterialsRef.current = selectedMaterials; }, [selectedMaterials]);
|
||||
|
||||
// Build a cache key from materials + duration
|
||||
// Computed: primary material
|
||||
const primaryMaterial = materials.find((m) => m.id === selectedMaterials[0]);
|
||||
|
||||
// Cache key
|
||||
const cacheKey = `${selectedMaterials.join(",")}_${audioDuration.toFixed(1)}`;
|
||||
const lsKey = storageKey ? `vigent_${storageKey}_timeline` : null;
|
||||
const lsKey = storageKey ? `vigent_${storageKey}_multicam` : null;
|
||||
|
||||
const initSegments = useCallback(() => {
|
||||
if (selectedMaterials.length === 0 || audioDuration <= 0) {
|
||||
setSegments([]);
|
||||
// Reset primary source range when primary material identity changes
|
||||
// (React render-time state adjustment pattern for derived state)
|
||||
if (selectedMaterials[0] !== prevPrimaryId) {
|
||||
setPrevPrimaryId(selectedMaterials[0]);
|
||||
setPrimarySourceStart(0);
|
||||
setPrimarySourceEnd(0);
|
||||
}
|
||||
|
||||
// Initialize / restore from localStorage
|
||||
const initInserts = useCallback(() => {
|
||||
if (selectedMaterials.length <= 1 || audioDuration <= 0) {
|
||||
setInserts([]);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -132,27 +102,28 @@ export const useTimelineEditor = ({
|
||||
try {
|
||||
const raw = localStorage.getItem(lsKey);
|
||||
if (raw) {
|
||||
const saved = JSON.parse(raw) as { key: string; segments: SegmentSnapshot[] };
|
||||
if (saved.key === cacheKey && saved.segments.length === selectedMaterials.length) {
|
||||
const allMatch = saved.segments.every(
|
||||
(s, i) => s.materialId === selectedMaterials[i] || saved.segments.some((ss) => ss.materialId === selectedMaterials[i])
|
||||
);
|
||||
if (allMatch) {
|
||||
const restored: TimelineSegment[] = saved.segments.map((s, i) => {
|
||||
const saved: MultiCamCache = JSON.parse(raw);
|
||||
if (saved.key === cacheKey) {
|
||||
// Validate all insert materialIds still exist
|
||||
const existingIds = new Set(materials.map((m) => m.id));
|
||||
const validInserts = saved.inserts.filter((s) => existingIds.has(s.materialId));
|
||||
if (validInserts.length === saved.inserts.length) {
|
||||
const restored: InsertSegment[] = validInserts.map((s, i) => {
|
||||
const mat = materials.find((m) => m.id === s.materialId);
|
||||
return {
|
||||
id: `seg-${i}-${Date.now()}`,
|
||||
id: `ins-${i}-${Date.now()}`,
|
||||
materialId: s.materialId,
|
||||
materialName: mat?.scene || mat?.name || s.materialId,
|
||||
start: 0,
|
||||
end: 0,
|
||||
start: s.start,
|
||||
end: s.end,
|
||||
sourceStart: s.sourceStart,
|
||||
sourceEnd: s.sourceEnd,
|
||||
color: COLORS[i % COLORS.length],
|
||||
};
|
||||
});
|
||||
setSegments(recalcPositions(restored, materials, audioDuration));
|
||||
restoredRef.current = true;
|
||||
setInserts(restored);
|
||||
setPrimarySourceStart(saved.primarySourceStart || 0);
|
||||
setPrimarySourceEnd(saved.primarySourceEnd || 0);
|
||||
return;
|
||||
}
|
||||
}
|
||||
@@ -162,95 +133,315 @@ export const useTimelineEditor = ({
|
||||
}
|
||||
}
|
||||
|
||||
// Create fresh segments — positions derived by recalcPositions
|
||||
const newSegments: TimelineSegment[] = selectedMaterials.map((matId, i) => {
|
||||
const mat = materials.find((m) => m.id === matId);
|
||||
return {
|
||||
id: `seg-${i}-${Date.now()}`,
|
||||
materialId: matId,
|
||||
materialName: mat?.scene || mat?.name || matId,
|
||||
start: 0,
|
||||
end: 0,
|
||||
sourceStart: 0,
|
||||
sourceEnd: 0,
|
||||
color: COLORS[i % COLORS.length],
|
||||
};
|
||||
});
|
||||
|
||||
setSegments(recalcPositions(newSegments, materials, audioDuration));
|
||||
// Start fresh
|
||||
setInserts([]);
|
||||
setPrimarySourceStart(0);
|
||||
setPrimarySourceEnd(0);
|
||||
}, [audioDuration, materials, selectedMaterials, lsKey, cacheKey]);
|
||||
|
||||
// Auto-init when selectedMaterials, audioDuration, or material durations change
|
||||
// Auto-init when inputs change
|
||||
useEffect(() => {
|
||||
const durationsKey = selectedMaterials
|
||||
.map((id) => materials.find((m) => m.id === id)?.duration_sec ?? 0)
|
||||
.join(",");
|
||||
const key = `${selectedMaterials.join(",")}_${audioDuration}_${durationsKey}`;
|
||||
if (key !== prevKey.current) {
|
||||
prevKey.current = key;
|
||||
initSegments();
|
||||
// eslint-disable-next-line react-hooks/set-state-in-effect -- initialization on input change
|
||||
initInserts();
|
||||
}
|
||||
}, [selectedMaterials, audioDuration, durationsKey, initSegments]);
|
||||
}, [selectedMaterials, audioDuration, materials, initInserts]);
|
||||
|
||||
// Persist segments to localStorage on change (debounced)
|
||||
// Persist to localStorage (debounced)
|
||||
useEffect(() => {
|
||||
if (!lsKey || segments.length === 0) return;
|
||||
if (!lsKey || selectedMaterials.length <= 1) return;
|
||||
const timeout = setTimeout(() => {
|
||||
const snapshots: SegmentSnapshot[] = segments.map((s) => ({
|
||||
const snapshots: InsertSnapshot[] = inserts.map((s) => ({
|
||||
materialId: s.materialId,
|
||||
start: s.start,
|
||||
end: s.end,
|
||||
sourceStart: s.sourceStart,
|
||||
sourceEnd: s.sourceEnd,
|
||||
}));
|
||||
localStorage.setItem(lsKey, JSON.stringify({ key: cacheKey, segments: snapshots }));
|
||||
const cache: MultiCamCache = {
|
||||
key: cacheKey,
|
||||
inserts: snapshots,
|
||||
primarySourceStart,
|
||||
primarySourceEnd,
|
||||
};
|
||||
localStorage.setItem(lsKey, JSON.stringify(cache));
|
||||
}, 300);
|
||||
return () => clearTimeout(timeout);
|
||||
}, [segments, lsKey, cacheKey]);
|
||||
}, [inserts, primarySourceStart, primarySourceEnd, lsKey, cacheKey, selectedMaterials.length]);
|
||||
|
||||
const reorderSegments = useCallback(
|
||||
(fromIdx: number, toIdx: number) => {
|
||||
setSegments((prev) => {
|
||||
if (fromIdx < 0 || toIdx < 0 || fromIdx >= prev.length || toIdx >= prev.length) return prev;
|
||||
if (fromIdx === toIdx) return prev;
|
||||
const next = [...prev];
|
||||
// Move the segment: remove from old position, insert at new position
|
||||
const [moved] = next.splice(fromIdx, 1);
|
||||
next.splice(toIdx, 0, moved);
|
||||
return recalcPositions(next, materialsRef.current, audioDurationRef.current);
|
||||
});
|
||||
},
|
||||
[]
|
||||
);
|
||||
// Clean up inserts referencing removed materials
|
||||
useEffect(() => {
|
||||
const existingIds = new Set(selectedMaterials.slice(1));
|
||||
// eslint-disable-next-line react-hooks/set-state-in-effect -- cleanup stale references
|
||||
setInserts((prev) => {
|
||||
const filtered = prev.filter((ins) => existingIds.has(ins.materialId));
|
||||
return filtered.length !== prev.length ? filtered : prev;
|
||||
});
|
||||
}, [selectedMaterials]);
|
||||
|
||||
const setSourceRange = useCallback(
|
||||
(id: string, sourceStart: number, sourceEnd: number) => {
|
||||
setSegments((prev) => {
|
||||
const updated = prev.map((s) => (s.id === id ? { ...s, sourceStart, sourceEnd } : s));
|
||||
return recalcPositions(updated, materialsRef.current, audioDurationRef.current);
|
||||
});
|
||||
},
|
||||
[]
|
||||
);
|
||||
// ── Operations ──
|
||||
|
||||
const addInsert = useCallback((materialId: string): AddInsertResult => {
|
||||
const currentInserts = inserts;
|
||||
const duration = audioDurationRef.current;
|
||||
|
||||
if (currentInserts.length >= MAX_INSERTS) return "limit";
|
||||
if (duration <= 0) return "no_space";
|
||||
|
||||
const mat = materialsRef.current.find((m) => m.id === materialId);
|
||||
const sorted = [...currentInserts].sort((a, b) => a.start - b.start);
|
||||
|
||||
// Find first gap that can fit DEFAULT_INSERT_DURATION
|
||||
let bestStart = -1;
|
||||
let prevEnd = 0;
|
||||
for (const ins of sorted) {
|
||||
if (ins.start - prevEnd >= DEFAULT_INSERT_DURATION + MIN_GAP) {
|
||||
bestStart = prevEnd + MIN_GAP;
|
||||
break;
|
||||
}
|
||||
prevEnd = ins.end;
|
||||
}
|
||||
// Check trailing gap
|
||||
if (bestStart < 0 && duration - prevEnd >= DEFAULT_INSERT_DURATION + MIN_GAP) {
|
||||
bestStart = prevEnd + MIN_GAP;
|
||||
}
|
||||
|
||||
if (bestStart < 0) return "no_space";
|
||||
|
||||
const newInsert: InsertSegment = {
|
||||
id: `ins-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`,
|
||||
materialId,
|
||||
materialName: mat?.scene || mat?.name || materialId,
|
||||
start: bestStart,
|
||||
end: Math.min(bestStart + DEFAULT_INSERT_DURATION, duration),
|
||||
sourceStart: 0,
|
||||
sourceEnd: 0,
|
||||
color: COLORS[currentInserts.length % COLORS.length],
|
||||
};
|
||||
|
||||
setInserts((prev) => [...prev, newInsert]);
|
||||
return "ok";
|
||||
}, [inserts]);
|
||||
|
||||
const removeInsert = useCallback((id: string) => {
|
||||
setInserts((prev) => prev.filter((ins) => ins.id !== id));
|
||||
}, []);
|
||||
|
||||
const moveInsert = useCallback((id: string, newStart: number) => {
|
||||
setInserts((prev) => {
|
||||
const duration = audioDurationRef.current;
|
||||
const target = prev.find((ins) => ins.id === id);
|
||||
if (!target) return prev;
|
||||
|
||||
const len = target.end - target.start;
|
||||
let clampedStart = Math.max(0, Math.min(newStart, duration - len));
|
||||
let clampedEnd = clampedStart + len;
|
||||
|
||||
// Prevent overlap with other inserts
|
||||
const others = prev.filter((ins) => ins.id !== id).sort((a, b) => a.start - b.start);
|
||||
for (const other of others) {
|
||||
if (clampedEnd > other.start && clampedStart < other.end) {
|
||||
// Try pushing to right of blocker
|
||||
const rightStart = other.end + 0.1;
|
||||
if (rightStart + len <= duration) {
|
||||
clampedStart = rightStart;
|
||||
clampedEnd = clampedStart + len;
|
||||
} else {
|
||||
// Try pushing to left of blocker
|
||||
const leftEnd = other.start - 0.1;
|
||||
if (leftEnd - len >= 0) {
|
||||
clampedEnd = leftEnd;
|
||||
clampedStart = clampedEnd - len;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return prev.map((ins) =>
|
||||
ins.id === id ? { ...ins, start: clampedStart, end: clampedEnd } : ins
|
||||
);
|
||||
});
|
||||
}, []);
|
||||
|
||||
const resizeInsert = useCallback((id: string, newEnd: number) => {
|
||||
setInserts((prev) => {
|
||||
const duration = audioDurationRef.current;
|
||||
const target = prev.find((ins) => ins.id === id);
|
||||
if (!target) return prev;
|
||||
|
||||
const minLen = 0.5;
|
||||
let clamped = Math.max(target.start + minLen, Math.min(newEnd, duration));
|
||||
|
||||
// Prevent overlap with next insert
|
||||
const others = prev.filter((ins) => ins.id !== id).sort((a, b) => a.start - b.start);
|
||||
for (const other of others) {
|
||||
if (other.start > target.start && clamped > other.start - 0.1) {
|
||||
clamped = other.start - 0.1;
|
||||
}
|
||||
}
|
||||
|
||||
return prev.map((ins) =>
|
||||
ins.id === id ? { ...ins, end: Math.max(clamped, target.start + minLen) } : ins
|
||||
);
|
||||
});
|
||||
}, []);
|
||||
|
||||
const setInsertSourceRange = useCallback((id: string, sourceStart: number, sourceEnd: number) => {
|
||||
setInserts((prev) =>
|
||||
prev.map((ins) => (ins.id === id ? { ...ins, sourceStart, sourceEnd } : ins))
|
||||
);
|
||||
}, []);
|
||||
|
||||
const setPrimarySourceRange = useCallback((sourceStart: number, sourceEnd: number) => {
|
||||
setPrimarySourceStart(sourceStart);
|
||||
setPrimarySourceEnd(sourceEnd);
|
||||
}, []);
|
||||
|
||||
// ── Serialization ──
|
||||
|
||||
const toCustomAssignments = useCallback((): CustomAssignment[] => {
|
||||
const mats = materialsRef.current;
|
||||
const selMats = selectedMaterialsRef.current;
|
||||
const duration = audioDurationRef.current;
|
||||
return segments
|
||||
.filter((seg) => seg.start < duration)
|
||||
.map((seg) => {
|
||||
const mat = materialsRef.current.find((m) => m.id === seg.materialId);
|
||||
return {
|
||||
material_path: mat?.path || seg.materialId,
|
||||
start: seg.start,
|
||||
end: seg.end,
|
||||
source_start: seg.sourceStart,
|
||||
source_end: seg.sourceEnd > seg.sourceStart ? seg.sourceEnd : undefined,
|
||||
};
|
||||
});
|
||||
}, [segments]);
|
||||
|
||||
if (duration <= 0 || selMats.length === 0) return [];
|
||||
|
||||
const primaryMat = mats.find((m) => m.id === selMats[0]);
|
||||
if (!primaryMat) return [];
|
||||
const primaryPath = primaryMat.path;
|
||||
const primaryDuration = primaryMat.duration_sec ?? 0;
|
||||
|
||||
// Single material mode: only emit assignment if user has set a source range
|
||||
if (selMats.length === 1) {
|
||||
if (primarySourceStart > 0 || primarySourceEnd > 0) {
|
||||
return [{
|
||||
material_path: primaryPath,
|
||||
start: 0,
|
||||
end: duration,
|
||||
source_start: primarySourceStart,
|
||||
source_end: primarySourceEnd > primarySourceStart ? primarySourceEnd : undefined,
|
||||
}];
|
||||
}
|
||||
return [];
|
||||
}
|
||||
|
||||
// Multi-camera mode: build assignments with gap splitting
|
||||
return buildAssignments(
|
||||
primaryPath,
|
||||
primaryDuration,
|
||||
primarySourceStart,
|
||||
primarySourceEnd,
|
||||
inserts,
|
||||
duration,
|
||||
mats,
|
||||
);
|
||||
}, [inserts, primarySourceStart, primarySourceEnd]);
|
||||
|
||||
return {
|
||||
segments,
|
||||
initSegments,
|
||||
reorderSegments,
|
||||
setSourceRange,
|
||||
// State
|
||||
inserts,
|
||||
setInserts,
|
||||
primaryMaterial,
|
||||
primarySourceStart,
|
||||
primarySourceEnd,
|
||||
// Operations
|
||||
addInsert,
|
||||
removeInsert,
|
||||
moveInsert,
|
||||
resizeInsert,
|
||||
setInsertSourceRange,
|
||||
setPrimarySourceRange,
|
||||
// Serialization
|
||||
toCustomAssignments,
|
||||
};
|
||||
};
|
||||
|
||||
// ── buildAssignments: gap-filling + boundary-splitting ──
|
||||
|
||||
function buildAssignments(
|
||||
primaryPath: string,
|
||||
primaryDuration: number,
|
||||
pSourceStart: number,
|
||||
pSourceEnd: number,
|
||||
inserts: InsertSegment[],
|
||||
audioDuration: number,
|
||||
materials: Material[],
|
||||
): CustomAssignment[] {
|
||||
const assignments: CustomAssignment[] = [];
|
||||
const sorted = [...inserts].sort((a, b) => a.start - b.start);
|
||||
|
||||
// Primary material effective play range
|
||||
const clipStart = pSourceStart;
|
||||
const clipEnd = pSourceEnd > pSourceStart ? pSourceEnd : primaryDuration;
|
||||
const effective = clipEnd - clipStart;
|
||||
|
||||
let cursor = 0;
|
||||
let primaryAccum = 0;
|
||||
|
||||
function addPrimaryGap(gapStart: number, gapEnd: number) {
|
||||
if (gapEnd - gapStart < 0.05) return;
|
||||
|
||||
// No valid effective range: single segment from 0 (graceful degradation)
|
||||
if (effective <= 0) {
|
||||
assignments.push({
|
||||
material_path: primaryPath,
|
||||
start: gapStart,
|
||||
end: gapEnd,
|
||||
source_start: 0,
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
let remaining = gapEnd - gapStart;
|
||||
let segStart = gapStart;
|
||||
const EPSILON = 0.01;
|
||||
|
||||
while (remaining > 0.05) {
|
||||
const posInClip = primaryAccum % effective;
|
||||
const sourceStart = clipStart + posInClip;
|
||||
const availableInClip = effective - posInClip;
|
||||
const segDuration = Math.min(remaining, availableInClip);
|
||||
|
||||
if (segDuration < EPSILON) break;
|
||||
|
||||
assignments.push({
|
||||
material_path: primaryPath,
|
||||
start: segStart,
|
||||
end: segStart + segDuration,
|
||||
source_start: sourceStart,
|
||||
source_end: pSourceEnd > pSourceStart ? pSourceEnd : undefined,
|
||||
});
|
||||
|
||||
primaryAccum += segDuration;
|
||||
segStart += segDuration;
|
||||
remaining -= segDuration;
|
||||
}
|
||||
}
|
||||
|
||||
for (const insert of sorted) {
|
||||
// Primary gap before this insert
|
||||
addPrimaryGap(cursor, insert.start);
|
||||
|
||||
// Insert segment
|
||||
const mat = materials.find((m) => m.id === insert.materialId);
|
||||
assignments.push({
|
||||
material_path: mat?.path || insert.materialId,
|
||||
start: insert.start,
|
||||
end: insert.end,
|
||||
source_start: insert.sourceStart,
|
||||
source_end: insert.sourceEnd > insert.sourceStart ? insert.sourceEnd : undefined,
|
||||
});
|
||||
|
||||
cursor = insert.end;
|
||||
}
|
||||
|
||||
// Trailing primary gap
|
||||
addPrimaryGap(cursor, audioDuration);
|
||||
|
||||
return assignments;
|
||||
}
|
||||
|
||||
@@ -5,9 +5,11 @@ import { useRouter } from "next/navigation";
|
||||
import { RefreshCw } from "lucide-react";
|
||||
import VideoPreviewModal from "@/components/VideoPreviewModal";
|
||||
import ScriptExtractionModal from "./ScriptExtractionModal";
|
||||
import ScriptLearningModal from "./ScriptLearningModal";
|
||||
import RewriteModal from "./RewriteModal";
|
||||
import { useHomeController } from "@/features/home/model/useHomeController";
|
||||
import { resolveMediaUrl } from "@/shared/lib/media";
|
||||
import { toast } from "sonner";
|
||||
import { BgmPanel } from "@/features/home/ui/BgmPanel";
|
||||
import { GenerateActionBar } from "@/features/home/ui/GenerateActionBar";
|
||||
import { HistoryList } from "@/features/home/ui/HistoryList";
|
||||
@@ -53,6 +55,8 @@ export function HomePage() {
|
||||
setText,
|
||||
extractModalOpen,
|
||||
setExtractModalOpen,
|
||||
learningModalOpen,
|
||||
setLearningModalOpen,
|
||||
rewriteModalOpen,
|
||||
setRewriteModalOpen,
|
||||
handleGenerateMeta,
|
||||
@@ -171,9 +175,19 @@ export function HomePage() {
|
||||
setSpeed,
|
||||
emotion,
|
||||
setEmotion,
|
||||
timelineSegments,
|
||||
reorderSegments,
|
||||
setSourceRange,
|
||||
// Multi-camera timeline
|
||||
inserts,
|
||||
timelinePrimaryMaterial,
|
||||
primarySourceStart,
|
||||
primarySourceEnd,
|
||||
insertCandidates,
|
||||
addInsert,
|
||||
removeInsert,
|
||||
moveInsert,
|
||||
resizeInsert,
|
||||
setInsertSourceRange,
|
||||
setPrimarySourceRange,
|
||||
handleSetPrimary,
|
||||
clipTrimmerOpen,
|
||||
setClipTrimmerOpen,
|
||||
clipTrimmerSegmentId,
|
||||
@@ -198,10 +212,27 @@ export function HomePage() {
|
||||
return () => clearTimeout(timer);
|
||||
}, []);
|
||||
|
||||
const clipTrimmerSegment = useMemo(
|
||||
() => timelineSegments.find((s) => s.id === clipTrimmerSegmentId) ?? null,
|
||||
[timelineSegments, clipTrimmerSegmentId]
|
||||
);
|
||||
// ClipTrimmer: construct segment from either primary or an insert
|
||||
const clipTrimmerSegment = useMemo(() => {
|
||||
if (!clipTrimmerSegmentId) return null;
|
||||
// Check if it's the primary material
|
||||
if (clipTrimmerSegmentId === "primary" && timelinePrimaryMaterial) {
|
||||
return {
|
||||
id: "primary",
|
||||
materialId: timelinePrimaryMaterial.id,
|
||||
materialName: timelinePrimaryMaterial.scene || timelinePrimaryMaterial.name || "",
|
||||
start: 0,
|
||||
end: selectedAudio?.duration_sec ?? 0,
|
||||
sourceStart: primarySourceStart,
|
||||
sourceEnd: primarySourceEnd,
|
||||
color: "#8b5cf6",
|
||||
};
|
||||
}
|
||||
// Check inserts
|
||||
const insert = inserts.find((i) => i.id === clipTrimmerSegmentId);
|
||||
if (insert) return insert;
|
||||
return null;
|
||||
}, [clipTrimmerSegmentId, timelinePrimaryMaterial, inserts, selectedAudio, primarySourceStart, primarySourceEnd]);
|
||||
|
||||
const clipTrimmerMaterialUrl = useMemo(() => {
|
||||
if (!clipTrimmerSegment) return null;
|
||||
@@ -222,6 +253,7 @@ export function HomePage() {
|
||||
text={text}
|
||||
onChangeText={setText}
|
||||
onOpenExtractModal={() => setExtractModalOpen(true)}
|
||||
onOpenLearningModal={() => setLearningModalOpen(true)}
|
||||
onOpenRewriteModal={() => setRewriteModalOpen(true)}
|
||||
onTranslate={handleTranslate}
|
||||
isTranslating={isTranslating}
|
||||
@@ -329,6 +361,7 @@ export function HomePage() {
|
||||
onDeleteMaterial={deleteMaterial}
|
||||
onClearUploadError={() => setUploadError(null)}
|
||||
registerMaterialRef={registerMaterialRef}
|
||||
onSetPrimary={handleSetPrimary}
|
||||
/>
|
||||
<div className="border-t border-white/10 my-4" />
|
||||
<div className="relative">
|
||||
@@ -343,15 +376,28 @@ export function HomePage() {
|
||||
embedded
|
||||
audioDuration={selectedAudio?.duration_sec ?? 0}
|
||||
audioUrl={selectedAudio ? (resolveMediaUrl(selectedAudio.path) || "") : ""}
|
||||
segments={timelineSegments}
|
||||
materials={materials}
|
||||
outputAspectRatio={outputAspectRatio}
|
||||
onOutputAspectRatioChange={setOutputAspectRatio}
|
||||
onReorderSegment={reorderSegments}
|
||||
onClickSegment={(seg) => {
|
||||
setClipTrimmerSegmentId(seg.id);
|
||||
primaryMaterial={timelinePrimaryMaterial}
|
||||
inserts={inserts}
|
||||
insertCandidates={insertCandidates}
|
||||
onAddInsert={(materialId) => {
|
||||
const result = addInsert(materialId);
|
||||
if (result === "limit") toast.error("最多添加 10 个插入片段");
|
||||
else if (result === "no_space") toast.error("时间轴空间不足，无法再添加插入");
|
||||
}}
|
||||
onRemoveInsert={removeInsert}
|
||||
onMoveInsert={moveInsert}
|
||||
onClickInsert={(insert) => {
|
||||
setClipTrimmerSegmentId(insert.id);
|
||||
setClipTrimmerOpen(true);
|
||||
}}
|
||||
onClickPrimary={() => {
|
||||
setClipTrimmerSegmentId("primary");
|
||||
setClipTrimmerOpen(true);
|
||||
}}
|
||||
primarySourceStart={primarySourceStart}
|
||||
primarySourceEnd={primarySourceEnd}
|
||||
outputAspectRatio={outputAspectRatio}
|
||||
onOutputAspectRatioChange={setOutputAspectRatio}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
@@ -514,13 +560,28 @@ export function HomePage() {
|
||||
onApply={(newText) => setText(newText)}
|
||||
/>
|
||||
|
||||
<ScriptLearningModal
|
||||
isOpen={learningModalOpen}
|
||||
onClose={() => setLearningModalOpen(false)}
|
||||
onApply={(nextText) => setText(nextText)}
|
||||
/>
|
||||
|
||||
<ClipTrimmer
|
||||
isOpen={clipTrimmerOpen}
|
||||
segment={clipTrimmerSegment}
|
||||
materialUrl={clipTrimmerMaterialUrl}
|
||||
onConfirm={(sourceStart, sourceEnd) => {
|
||||
if (clipTrimmerSegmentId) {
|
||||
setSourceRange(clipTrimmerSegmentId, sourceStart, sourceEnd);
|
||||
if (clipTrimmerSegmentId === "primary") {
|
||||
setPrimarySourceRange(sourceStart, sourceEnd);
|
||||
} else if (clipTrimmerSegmentId) {
|
||||
setInsertSourceRange(clipTrimmerSegmentId, sourceStart, sourceEnd);
|
||||
// Sync timeline duration to match trimmed source duration
|
||||
if (sourceEnd > sourceStart) {
|
||||
const ins = inserts.find((i) => i.id === clipTrimmerSegmentId);
|
||||
if (ins) {
|
||||
resizeInsert(clipTrimmerSegmentId, ins.start + (sourceEnd - sourceStart));
|
||||
}
|
||||
}
|
||||
}
|
||||
setClipTrimmerOpen(false);
|
||||
}}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { type ChangeEvent, type MouseEvent, useCallback, useMemo, useRef, useState } from "react";
|
||||
import { Upload, RefreshCw, Eye, Trash2, X, Pencil, Check, Search, ChevronDown } from "lucide-react";
|
||||
import { Upload, RefreshCw, Eye, Trash2, X, Pencil, Check, Search, ChevronDown, Crown } from "lucide-react";
|
||||
import type { Material } from "@/shared/types/material";
|
||||
import { SelectPopover } from "@/shared/ui/SelectPopover";
|
||||
|
||||
@@ -26,6 +26,7 @@ interface MaterialSelectorProps {
|
||||
onDeleteMaterial: (id: string) => void;
|
||||
onClearUploadError: () => void;
|
||||
registerMaterialRef: (id: string, element: HTMLDivElement | null) => void;
|
||||
onSetPrimary?: (materialId: string) => void;
|
||||
embedded?: boolean;
|
||||
}
|
||||
|
||||
@@ -52,6 +53,7 @@ export function MaterialSelector({
|
||||
onDeleteMaterial,
|
||||
onClearUploadError,
|
||||
registerMaterialRef,
|
||||
onSetPrimary,
|
||||
embedded = false,
|
||||
}: MaterialSelectorProps) {
|
||||
const [materialFilter, setMaterialFilter] = useState("");
|
||||
@@ -280,12 +282,33 @@ export function MaterialSelector({
|
||||
disabled={isFull && !isSelected}
|
||||
className="min-w-0 flex-1 text-left"
|
||||
>
|
||||
<span className="block truncate text-sm text-white">{m.scene || m.name}</span>
|
||||
<span className="flex items-center gap-1.5">
|
||||
<span className="block truncate text-sm text-white">{m.scene || m.name}</span>
|
||||
{isSelected && selectedMaterials[0] === m.id && selectedMaterials.length > 1 && (
|
||||
<span className="shrink-0 text-[9px] px-1 py-0.5 rounded bg-purple-500/30 text-purple-300 border border-purple-500/40">主素材</span>
|
||||
)}
|
||||
{isSelected && selectedMaterials[0] !== m.id && (
|
||||
<span className="shrink-0 text-[9px] px-1 py-0.5 rounded bg-white/10 text-gray-400 border border-white/10">可插入</span>
|
||||
)}
|
||||
</span>
|
||||
<span className="mt-0.5 block text-xs text-gray-400">{m.size_mb.toFixed(1)} MB</span>
|
||||
</button>
|
||||
)}
|
||||
|
||||
<div className="flex items-center gap-2 pl-2">
|
||||
{isSelected && selectedMaterials[0] !== m.id && onSetPrimary && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
onSetPrimary(m.id);
|
||||
}}
|
||||
className="p-1 text-gray-400 hover:text-amber-300"
|
||||
title="设为主素材"
|
||||
>
|
||||
<Crown className="h-4 w-4" />
|
||||
</button>
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => {
|
||||
|
||||
@@ -3,6 +3,7 @@ import { Loader2, Sparkles } from "lucide-react";
|
||||
import api from "@/shared/api/axios";
|
||||
import { ApiResponse, unwrap } from "@/shared/api/types";
|
||||
import { AppModal, AppModalHeader } from "@/shared/ui/AppModal";
|
||||
import { toast } from "sonner";
|
||||
|
||||
const CUSTOM_PROMPT_KEY = "vigent_rewriteCustomPrompt";
|
||||
|
||||
@@ -78,10 +79,52 @@ export default function RewriteModal({
|
||||
onClose();
|
||||
};
|
||||
|
||||
const handleRetry = () => {
|
||||
setRewrittenText("");
|
||||
setError(null);
|
||||
};
|
||||
const handleRetry = () => {
|
||||
setRewrittenText("");
|
||||
setError(null);
|
||||
};
|
||||
|
||||
const fallbackCopyTextToClipboard = useCallback((text: string) => {
|
||||
const textArea = document.createElement("textarea");
|
||||
textArea.value = text;
|
||||
textArea.style.top = "0";
|
||||
textArea.style.left = "0";
|
||||
textArea.style.position = "fixed";
|
||||
textArea.style.opacity = "0";
|
||||
|
||||
document.body.appendChild(textArea);
|
||||
textArea.focus();
|
||||
textArea.select();
|
||||
|
||||
try {
|
||||
const successful = document.execCommand("copy");
|
||||
if (successful) {
|
||||
toast.success("已复制到剪贴板");
|
||||
} else {
|
||||
toast.error("复制失败，请手动复制");
|
||||
}
|
||||
} catch {
|
||||
toast.error("复制失败，请手动复制");
|
||||
}
|
||||
|
||||
document.body.removeChild(textArea);
|
||||
}, []);
|
||||
|
||||
const handleCopy = useCallback((text: string) => {
|
||||
if (!text.trim()) return;
|
||||
if (navigator.clipboard && window.isSecureContext) {
|
||||
navigator.clipboard
|
||||
.writeText(text)
|
||||
.then(() => {
|
||||
toast.success("已复制到剪贴板");
|
||||
})
|
||||
.catch(() => {
|
||||
fallbackCopyTextToClipboard(text);
|
||||
});
|
||||
} else {
|
||||
fallbackCopyTextToClipboard(text);
|
||||
}
|
||||
}, [fallbackCopyTextToClipboard]);
|
||||
|
||||
if (!isOpen) return null;
|
||||
|
||||
@@ -143,56 +186,63 @@ export default function RewriteModal({
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Rewritten result */}
|
||||
{rewrittenText && (
|
||||
<>
|
||||
<div className="space-y-2">
|
||||
<div className="flex justify-between items-center">
|
||||
<h4 className="font-semibold text-purple-300 flex items-center gap-2">
|
||||
<Sparkles className="h-4 w-4" />
|
||||
AI 改写结果
|
||||
</h4>
|
||||
<button
|
||||
onClick={handleApply}
|
||||
className="text-xs bg-gradient-to-r from-purple-600 to-pink-600 hover:from-purple-500 hover:to-pink-500 text-white px-3 py-1.5 rounded-lg transition-colors shadow-sm"
|
||||
>
|
||||
使用此结果
|
||||
</button>
|
||||
</div>
|
||||
<div className="bg-purple-900/10 border border-purple-500/20 rounded-xl p-4 max-h-60 overflow-y-auto hide-scrollbar">
|
||||
<p className="text-gray-200 text-sm leading-relaxed whitespace-pre-wrap">
|
||||
{rewrittenText}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="space-y-2">
|
||||
<div className="flex justify-between items-center">
|
||||
<h4 className="font-semibold text-gray-400 flex items-center gap-2">
|
||||
📝 原文对比
|
||||
</h4>
|
||||
<button
|
||||
onClick={onClose}
|
||||
className="text-xs bg-white/10 hover:bg-white/20 text-white px-3 py-1.5 rounded-lg transition-colors"
|
||||
>
|
||||
保留原文
|
||||
</button>
|
||||
</div>
|
||||
<div className="bg-white/5 border border-white/10 rounded-xl p-4 max-h-40 overflow-y-auto hide-scrollbar">
|
||||
<p className="text-gray-400 text-sm leading-relaxed whitespace-pre-wrap">
|
||||
{originalText}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<button
|
||||
onClick={handleRetry}
|
||||
className="w-full py-2.5 px-4 bg-white/10 hover:bg-white/20 text-white rounded-xl transition-colors"
|
||||
>
|
||||
重新改写
|
||||
</button>
|
||||
</>
|
||||
)}
|
||||
{/* Rewritten result */}
|
||||
{rewrittenText && (
|
||||
<>
|
||||
<div className="space-y-2">
|
||||
<div className="flex justify-between items-center">
|
||||
<h4 className="font-semibold text-purple-300 flex items-center gap-2">
|
||||
<Sparkles className="h-4 w-4" />
|
||||
AI 改写结果
|
||||
</h4>
|
||||
<span className="text-xs text-gray-400">{rewrittenText.length} 字</span>
|
||||
</div>
|
||||
<div className="bg-purple-900/10 border border-purple-500/20 rounded-xl p-4 max-h-60 overflow-y-auto hide-scrollbar">
|
||||
<p className="text-gray-200 text-sm leading-relaxed whitespace-pre-wrap">
|
||||
{rewrittenText}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="space-y-2">
|
||||
<h4 className="font-semibold text-gray-400 flex items-center gap-2">
|
||||
📝 原文对比
|
||||
</h4>
|
||||
<div className="bg-white/5 border border-white/10 rounded-xl p-4 max-h-40 overflow-y-auto hide-scrollbar">
|
||||
<p className="text-gray-400 text-sm leading-relaxed whitespace-pre-wrap">
|
||||
{originalText}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-2 sm:grid-cols-4 gap-2">
|
||||
<button
|
||||
onClick={handleApply}
|
||||
className="py-2.5 px-3 bg-gradient-to-r from-purple-600 to-pink-600 hover:from-purple-500 hover:to-pink-500 text-white rounded-lg transition-colors text-sm"
|
||||
>
|
||||
填入文案
|
||||
</button>
|
||||
<button
|
||||
onClick={() => handleCopy(rewrittenText)}
|
||||
className="py-2.5 px-3 bg-white/10 hover:bg-white/20 text-white rounded-lg transition-colors text-sm"
|
||||
>
|
||||
复制
|
||||
</button>
|
||||
<button
|
||||
onClick={handleRetry}
|
||||
className="py-2.5 px-3 bg-white/10 hover:bg-white/20 text-white rounded-lg transition-colors text-sm"
|
||||
>
|
||||
重新生成
|
||||
</button>
|
||||
<button
|
||||
onClick={onClose}
|
||||
className="py-2.5 px-3 bg-white/10 hover:bg-white/20 text-white rounded-lg transition-colors text-sm"
|
||||
>
|
||||
保留原文
|
||||
</button>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
</AppModal>
|
||||
);
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { useCallback, useEffect, useRef, useState } from "react";
|
||||
import { FileText, History, Languages, Loader2, Maximize2, RotateCcw, Save, Sparkles, Trash2 } from "lucide-react";
|
||||
import { FileText, GraduationCap, History, Languages, Loader2, Maximize2, RotateCcw, Save, Sparkles, Trash2 } from "lucide-react";
|
||||
import type { SavedScript } from "@/features/home/model/useSavedScripts";
|
||||
import { AppModal, AppModalHeader } from "@/shared/ui/AppModal";
|
||||
|
||||
@@ -19,6 +19,7 @@ interface ScriptEditorProps {
|
||||
text: string;
|
||||
onChangeText: (value: string) => void;
|
||||
onOpenExtractModal: () => void;
|
||||
onOpenLearningModal: () => void;
|
||||
onOpenRewriteModal: () => void;
|
||||
onTranslate: (targetLang: string) => void;
|
||||
isTranslating: boolean;
|
||||
@@ -34,6 +35,7 @@ export function ScriptEditor({
|
||||
text,
|
||||
onChangeText,
|
||||
onOpenExtractModal,
|
||||
onOpenLearningModal,
|
||||
onOpenRewriteModal,
|
||||
onTranslate,
|
||||
isTranslating,
|
||||
@@ -146,6 +148,13 @@ export function ScriptEditor({
|
||||
<FileText className="h-3.5 w-3.5" />
|
||||
文案提取助手
|
||||
</button>
|
||||
<button
|
||||
onClick={onOpenLearningModal}
|
||||
className={`${actionBtnBase} bg-gradient-to-r from-blue-600 to-cyan-600 hover:from-blue-500 hover:to-cyan-500 text-white`}
|
||||
>
|
||||
<GraduationCap className="h-3.5 w-3.5" />
|
||||
文案深度学习
|
||||
</button>
|
||||
<div className="relative" ref={langMenuRef}>
|
||||
<button
|
||||
onClick={() => setShowLangMenu((prev) => !prev)}
|
||||
|
||||
@@ -228,43 +228,47 @@ export default function ScriptExtractionModal({
|
||||
)}
|
||||
|
||||
{step === "result" && (
|
||||
<div className="space-y-6">
|
||||
<div className="space-y-2">
|
||||
<div className="flex justify-between items-center">
|
||||
<h4 className="font-semibold text-gray-300 flex items-center gap-2">
|
||||
🎙️ 识别结果
|
||||
</h4>
|
||||
<div className="flex items-center gap-2">
|
||||
{onApply && (
|
||||
<button
|
||||
onClick={() => handleApplyAndClose(script)}
|
||||
className="text-xs bg-gradient-to-r from-purple-600 to-pink-600 hover:from-purple-500 hover:to-pink-500 text-white px-3 py-1.5 rounded-lg transition-colors flex items-center gap-1 shadow-sm"
|
||||
>
|
||||
📥 填入
|
||||
</button>
|
||||
)}
|
||||
<button
|
||||
onClick={() => copyToClipboard(script)}
|
||||
className="text-xs bg-white/10 hover:bg-white/20 text-white px-3 py-1.5 rounded-lg transition-colors"
|
||||
>
|
||||
复制
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div className="bg-white/5 border border-white/10 rounded-xl p-4 max-h-60 overflow-y-auto hide-scrollbar">
|
||||
<p className="text-gray-200 text-sm leading-relaxed whitespace-pre-wrap">
|
||||
{script}
|
||||
</p>
|
||||
</div>
|
||||
<div className="space-y-5">
|
||||
<div className="flex justify-between items-center">
|
||||
<h4 className="font-semibold text-gray-300 flex items-center gap-2">
|
||||
🎙️ 识别结果
|
||||
</h4>
|
||||
<span className="text-xs text-gray-400">{script.length} 字</span>
|
||||
</div>
|
||||
|
||||
<div className="flex justify-center pt-4">
|
||||
<div className="bg-white/5 border border-white/10 rounded-xl p-4 max-h-72 overflow-y-auto hide-scrollbar">
|
||||
<p className="text-gray-200 text-sm leading-relaxed whitespace-pre-wrap">
|
||||
{script}
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div className={`grid ${onApply ? "grid-cols-2 sm:grid-cols-4" : "grid-cols-2 sm:grid-cols-3"} gap-2`}>
|
||||
{onApply && (
|
||||
<button
|
||||
onClick={() => handleApplyAndClose(script)}
|
||||
className="py-2.5 px-3 bg-gradient-to-r from-purple-600 to-pink-600 hover:from-purple-500 hover:to-pink-500 text-white rounded-lg transition-colors text-sm"
|
||||
>
|
||||
填入文案
|
||||
</button>
|
||||
)}
|
||||
<button
|
||||
onClick={() => copyToClipboard(script)}
|
||||
className="py-2.5 px-3 bg-white/10 hover:bg-white/20 text-white rounded-lg transition-colors text-sm"
|
||||
>
|
||||
复制
|
||||
</button>
|
||||
<button
|
||||
onClick={handleExtractNext}
|
||||
className="px-6 py-2 bg-white/10 hover:bg-white/20 text-white rounded-lg transition-colors"
|
||||
className="py-2.5 px-3 bg-white/10 hover:bg-white/20 text-white rounded-lg transition-colors text-sm"
|
||||
>
|
||||
提取下一个
|
||||
</button>
|
||||
<button
|
||||
onClick={onClose}
|
||||
className="py-2.5 px-3 bg-white/10 hover:bg-white/20 text-white rounded-lg transition-colors text-sm"
|
||||
>
|
||||
关闭
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
242
frontend/src/features/home/ui/ScriptLearningModal.tsx
Normal file
242
frontend/src/features/home/ui/ScriptLearningModal.tsx
Normal file
@@ -0,0 +1,242 @@
|
||||
"use client";
|
||||
|
||||
import { BookOpen, Sparkles } from "lucide-react";
|
||||
import { AppModal, AppModalHeader } from "@/shared/ui/AppModal";
|
||||
import { useScriptLearning } from "./script-learning/useScriptLearning";
|
||||
|
||||
interface ScriptLearningModalProps {
|
||||
isOpen: boolean;
|
||||
onClose: () => void;
|
||||
onApply?: (text: string) => void;
|
||||
}
|
||||
|
||||
const WORD_COUNT_MIN = 80;
|
||||
const WORD_COUNT_MAX = 1000;
|
||||
|
||||
export default function ScriptLearningModal({ isOpen, onClose, onApply }: ScriptLearningModalProps) {
|
||||
const {
|
||||
step,
|
||||
inputUrl,
|
||||
setInputUrl,
|
||||
topics,
|
||||
selectedTopic,
|
||||
setSelectedTopic,
|
||||
wordCount,
|
||||
setWordCount,
|
||||
generatedScript,
|
||||
error,
|
||||
analysisId,
|
||||
handleAnalyze,
|
||||
handleGenerate,
|
||||
handleRegenerate,
|
||||
backToInput,
|
||||
backToTopics,
|
||||
copyToClipboard,
|
||||
} = useScriptLearning({ isOpen });
|
||||
|
||||
if (!isOpen) return null;
|
||||
|
||||
const wordCountNum = Number(wordCount);
|
||||
const wordCountValid = Number.isInteger(wordCountNum)
|
||||
&& wordCountNum >= WORD_COUNT_MIN
|
||||
&& wordCountNum <= WORD_COUNT_MAX;
|
||||
const canGenerate = !!analysisId && !!selectedTopic && wordCountValid;
|
||||
|
||||
const handleApplyAndClose = () => {
|
||||
if (!generatedScript.trim()) return;
|
||||
onApply?.(generatedScript);
|
||||
onClose();
|
||||
};
|
||||
|
||||
return (
|
||||
<AppModal
|
||||
isOpen={isOpen}
|
||||
onClose={onClose}
|
||||
panelClassName="w-full max-w-2xl max-h-[90vh] rounded-2xl border border-white/10 bg-[#171821]/95 shadow-[0_24px_80px_rgba(0,0,0,0.55)] overflow-hidden flex flex-col"
|
||||
closeOnOverlay={false}
|
||||
closeOnEsc={false}
|
||||
>
|
||||
<AppModalHeader
|
||||
title="文案深度学习"
|
||||
icon={<BookOpen className="h-5 w-5 text-cyan-300" />}
|
||||
subtitle="分析博主近期选题风格并快速生成文案"
|
||||
onClose={onClose}
|
||||
/>
|
||||
|
||||
<div className="flex-1 overflow-y-auto p-6">
|
||||
{step === "input" && (
|
||||
<div className="space-y-5">
|
||||
<div className="space-y-2">
|
||||
<label className="text-sm text-gray-300">博主主页链接</label>
|
||||
<input
|
||||
type="text"
|
||||
value={inputUrl}
|
||||
onChange={(event) => setInputUrl(event.target.value)}
|
||||
placeholder="请粘贴抖音或B站博主主页链接..."
|
||||
className="w-full bg-black/20 border border-white/10 rounded-xl px-4 py-3 text-white placeholder-gray-500 focus:outline-none focus:border-cyan-500 transition-colors"
|
||||
/>
|
||||
<p className="text-xs text-gray-500">仅支持 https 链接，建议使用主页地址（非单条视频链接）</p>
|
||||
</div>
|
||||
|
||||
{error && (
|
||||
<div className="bg-red-500/10 border border-red-500/30 rounded-xl p-4">
|
||||
<p className="text-red-400 text-sm">{error}</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="flex gap-3 pt-1">
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setInputUrl("")}
|
||||
className="flex-1 py-3 px-4 bg-white/10 hover:bg-white/20 text-white rounded-xl transition-colors"
|
||||
>
|
||||
清空
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => void handleAnalyze()}
|
||||
disabled={!inputUrl.trim()}
|
||||
className="flex-1 py-3 px-4 bg-gradient-to-r from-blue-600 to-cyan-600 hover:from-blue-500 hover:to-cyan-500 disabled:opacity-50 disabled:cursor-not-allowed text-white rounded-xl transition-all font-medium shadow-lg"
|
||||
>
|
||||
开始分析
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{(step === "analyzing" || step === "generating") && (
|
||||
<div className="flex flex-col items-center justify-center py-20">
|
||||
<div className="relative w-20 h-20 mb-6">
|
||||
<div className="absolute inset-0 border-4 border-cyan-500/30 rounded-full" />
|
||||
<div className="absolute inset-0 border-4 border-t-cyan-500 rounded-full animate-spin" />
|
||||
</div>
|
||||
<h4 className="text-xl font-medium text-white mb-2">
|
||||
{step === "analyzing" ? "正在分析中..." : "正在生成中..."}
|
||||
</h4>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{step === "topics" && (
|
||||
<div className="space-y-5">
|
||||
<div className="bg-cyan-500/10 border border-cyan-500/30 rounded-xl p-3">
|
||||
<p className="text-cyan-200 text-sm">已完成深度学习，请选择热门话题。</p>
|
||||
</div>
|
||||
|
||||
<div className="space-y-2">
|
||||
<p className="text-sm text-gray-300">请选择一个话题</p>
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 gap-2">
|
||||
{topics.map((topic) => {
|
||||
const active = selectedTopic === topic;
|
||||
return (
|
||||
<button
|
||||
key={topic}
|
||||
type="button"
|
||||
onClick={() => setSelectedTopic(topic)}
|
||||
className={`text-left rounded-lg border px-3 py-2.5 text-sm transition-colors ${
|
||||
active
|
||||
? "border-cyan-400 bg-cyan-500/20 text-cyan-100"
|
||||
: "border-white/10 bg-white/5 text-gray-200 hover:border-white/20 hover:bg-white/10"
|
||||
}`}
|
||||
>
|
||||
{topic}
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="space-y-2">
|
||||
<label className="text-sm text-gray-300">目标字数</label>
|
||||
<input
|
||||
type="number"
|
||||
min={WORD_COUNT_MIN}
|
||||
max={WORD_COUNT_MAX}
|
||||
value={wordCount}
|
||||
onChange={(event) => setWordCount(event.target.value)}
|
||||
placeholder="请输入目标字数（80-1000），如 300"
|
||||
className="w-full bg-black/20 border border-white/10 rounded-xl px-4 py-3 text-white placeholder-gray-500 focus:outline-none focus:border-cyan-500 transition-colors"
|
||||
/>
|
||||
</div>
|
||||
|
||||
{error && (
|
||||
<div className="bg-red-500/10 border border-red-500/30 rounded-xl p-4">
|
||||
<p className="text-red-400 text-sm">{error}</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="flex gap-3 pt-1">
|
||||
<button
|
||||
type="button"
|
||||
onClick={backToInput}
|
||||
className="flex-1 py-3 px-4 bg-white/10 hover:bg-white/20 text-white rounded-xl transition-colors"
|
||||
>
|
||||
返回
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => void handleGenerate()}
|
||||
disabled={!canGenerate}
|
||||
className="flex-1 py-3 px-4 bg-gradient-to-r from-blue-600 to-cyan-600 hover:from-blue-500 hover:to-cyan-500 disabled:opacity-50 disabled:cursor-not-allowed text-white rounded-xl transition-all font-medium shadow-lg"
|
||||
>
|
||||
生成文案
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{step === "result" && (
|
||||
<div className="space-y-5">
|
||||
<div className="flex justify-between items-center">
|
||||
<h4 className="font-semibold text-cyan-200 flex items-center gap-2">
|
||||
<Sparkles className="h-4 w-4" />
|
||||
生成结果
|
||||
</h4>
|
||||
<span className="text-xs text-gray-400">{generatedScript.length} 字</span>
|
||||
</div>
|
||||
|
||||
<div className="bg-white/5 border border-white/10 rounded-xl p-4 max-h-72 overflow-y-auto hide-scrollbar">
|
||||
<p className="text-gray-200 text-sm leading-relaxed whitespace-pre-wrap">{generatedScript}</p>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-2 sm:grid-cols-4 gap-2">
|
||||
<button
|
||||
type="button"
|
||||
onClick={handleApplyAndClose}
|
||||
className="py-2.5 px-3 bg-gradient-to-r from-blue-600 to-cyan-600 hover:from-blue-500 hover:to-cyan-500 text-white rounded-lg transition-colors text-sm"
|
||||
>
|
||||
填入文案
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => copyToClipboard(generatedScript)}
|
||||
className="py-2.5 px-3 bg-white/10 hover:bg-white/20 text-white rounded-lg transition-colors text-sm"
|
||||
>
|
||||
复制
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => void handleRegenerate()}
|
||||
className="py-2.5 px-3 bg-white/10 hover:bg-white/20 text-white rounded-lg transition-colors text-sm"
|
||||
>
|
||||
重新生成
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={backToTopics}
|
||||
className="py-2.5 px-3 bg-white/10 hover:bg-white/20 text-white rounded-lg transition-colors text-sm"
|
||||
>
|
||||
换个话题
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{error && (
|
||||
<div className="bg-red-500/10 border border-red-500/30 rounded-xl p-4">
|
||||
<p className="text-red-400 text-sm">{error}</p>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</AppModal>
|
||||
);
|
||||
}
|
||||
@@ -1,19 +1,28 @@
|
||||
import { useEffect, useRef, useCallback, useState, useMemo } from "react";
|
||||
import { useEffect, useRef, useCallback, useState } from "react";
|
||||
import WaveSurfer from "wavesurfer.js";
|
||||
import { ChevronDown, GripVertical, Check } from "lucide-react";
|
||||
import type { TimelineSegment } from "@/features/home/model/useTimelineEditor";
|
||||
import { ChevronDown, Check, X, Plus } from "lucide-react";
|
||||
import type { InsertSegment } from "@/shared/types/timeline";
|
||||
import type { Material } from "@/shared/types/material";
|
||||
import { SelectPopover } from "@/shared/ui/SelectPopover";
|
||||
|
||||
interface TimelineEditorProps {
|
||||
audioDuration: number;
|
||||
audioUrl: string;
|
||||
segments: TimelineSegment[];
|
||||
materials: Material[];
|
||||
// Multi-camera props
|
||||
primaryMaterial: Material | undefined;
|
||||
inserts: InsertSegment[];
|
||||
insertCandidates: Material[];
|
||||
onAddInsert: (materialId: string) => void;
|
||||
onRemoveInsert: (id: string) => void;
|
||||
onMoveInsert: (id: string, newStart: number) => void;
|
||||
onClickInsert: (insert: InsertSegment) => void;
|
||||
onClickPrimary: () => void;
|
||||
// Single material: for ClipTrimmer compat, pass a synthetic TimelineSegment
|
||||
primarySourceStart: number;
|
||||
primarySourceEnd: number;
|
||||
// Shared
|
||||
outputAspectRatio: "9:16" | "16:9";
|
||||
onOutputAspectRatioChange: (ratio: "9:16" | "16:9") => void;
|
||||
onReorderSegment: (fromIdx: number, toIdx: number) => void;
|
||||
onClickSegment: (segment: TimelineSegment) => void;
|
||||
embedded?: boolean;
|
||||
}
|
||||
|
||||
@@ -26,12 +35,18 @@ function formatTime(sec: number): string {
|
||||
export function TimelineEditor({
|
||||
audioDuration,
|
||||
audioUrl,
|
||||
segments,
|
||||
materials,
|
||||
primaryMaterial,
|
||||
inserts,
|
||||
insertCandidates,
|
||||
onAddInsert,
|
||||
onRemoveInsert,
|
||||
onMoveInsert,
|
||||
onClickInsert,
|
||||
onClickPrimary,
|
||||
primarySourceStart,
|
||||
primarySourceEnd,
|
||||
outputAspectRatio,
|
||||
onOutputAspectRatioChange,
|
||||
onReorderSegment,
|
||||
onClickSegment,
|
||||
embedded = false,
|
||||
}: TimelineEditorProps) {
|
||||
const waveRef = useRef<HTMLDivElement>(null);
|
||||
@@ -39,18 +54,25 @@ export function TimelineEditor({
|
||||
const [waveReady, setWaveReady] = useState(false);
|
||||
const [isPlaying, setIsPlaying] = useState(false);
|
||||
|
||||
// Refs for high-frequency DOM updates (avoid 60fps re-renders)
|
||||
// Refs for high-frequency DOM updates
|
||||
const playheadRef = useRef<HTMLDivElement>(null);
|
||||
const timeRef = useRef<HTMLSpanElement>(null);
|
||||
const audioDurationRef = useRef(audioDuration);
|
||||
const timelineBarRef = useRef<HTMLDivElement>(null);
|
||||
|
||||
useEffect(() => {
|
||||
audioDurationRef.current = audioDuration;
|
||||
}, [audioDuration]);
|
||||
|
||||
// Drag-to-reorder state
|
||||
const [dragFromIdx, setDragFromIdx] = useState<number | null>(null);
|
||||
const [dragOverIdx, setDragOverIdx] = useState<number | null>(null);
|
||||
// Drag state for insert blocks (move only; duration editing unified to ClipTrimmer)
|
||||
const [dragId, setDragId] = useState<string | null>(null);
|
||||
const dragStartXRef = useRef(0);
|
||||
const dragStartValRef = useRef(0);
|
||||
const dragMovedRef = useRef(false);
|
||||
const DRAG_THRESHOLD = 5;
|
||||
|
||||
const isMultiCam = insertCandidates.length > 0 || inserts.length > 0;
|
||||
const hasPrimary = !!primaryMaterial;
|
||||
|
||||
// Aspect ratio options
|
||||
const ratioOptions = [
|
||||
@@ -60,14 +82,21 @@ export function TimelineEditor({
|
||||
const currentRatioLabel =
|
||||
ratioOptions.find((opt) => opt.value === outputAspectRatio)?.label ?? "竖屏 9:16";
|
||||
|
||||
// Primary material loop info
|
||||
const primaryDuration = primaryMaterial?.duration_sec ?? 0;
|
||||
const primaryEffective = primarySourceEnd > primarySourceStart
|
||||
? primarySourceEnd - primarySourceStart
|
||||
: primaryDuration;
|
||||
const loopCount = primaryEffective > 0 && audioDuration > 0
|
||||
? (audioDuration / primaryEffective)
|
||||
: 0;
|
||||
|
||||
// Create / recreate wavesurfer when audioUrl changes
|
||||
useEffect(() => {
|
||||
if (!waveRef.current || !audioUrl) return;
|
||||
|
||||
const playheadEl = playheadRef.current;
|
||||
const timeEl = timeRef.current;
|
||||
|
||||
// Destroy previous instance
|
||||
if (wsRef.current) {
|
||||
wsRef.current.destroy();
|
||||
wsRef.current = null;
|
||||
@@ -87,7 +116,6 @@ export function TimelineEditor({
|
||||
normalize: true,
|
||||
});
|
||||
|
||||
// Click waveform → seek + auto-play
|
||||
ws.on("interaction", () => ws.play());
|
||||
ws.on("play", () => setIsPlaying(true));
|
||||
ws.on("pause", () => setIsPlaying(false));
|
||||
@@ -95,7 +123,6 @@ export function TimelineEditor({
|
||||
setIsPlaying(false);
|
||||
if (playheadRef.current) playheadRef.current.style.display = "none";
|
||||
});
|
||||
// High-frequency: update playhead + time via refs (no React re-render)
|
||||
ws.on("timeupdate", (time: number) => {
|
||||
const dur = audioDurationRef.current;
|
||||
if (playheadRef.current && dur > 0) {
|
||||
@@ -119,7 +146,6 @@ export function TimelineEditor({
|
||||
};
|
||||
}, [audioUrl, waveReady]);
|
||||
|
||||
// Callback ref to detect when waveRef div mounts
|
||||
const waveCallbackRef = useCallback((node: HTMLDivElement | null) => {
|
||||
(waveRef as React.MutableRefObject<HTMLDivElement | null>).current = node;
|
||||
setWaveReady(!!node);
|
||||
@@ -129,43 +155,45 @@ export function TimelineEditor({
|
||||
wsRef.current?.playPause();
|
||||
}, []);
|
||||
|
||||
// Drag-to-reorder handlers
|
||||
const handleDragStart = useCallback((idx: number, e: React.DragEvent) => {
|
||||
setDragFromIdx(idx);
|
||||
e.dataTransfer.effectAllowed = "move";
|
||||
e.dataTransfer.setData("text/plain", String(idx));
|
||||
}, []);
|
||||
// ── Insert block pointer handlers (move only) ──
|
||||
|
||||
const handleDragOver = useCallback((idx: number, e: React.DragEvent) => {
|
||||
const getTimeFromClientX = useCallback((clientX: number): number => {
|
||||
if (!timelineBarRef.current || audioDuration <= 0) return 0;
|
||||
const rect = timelineBarRef.current.getBoundingClientRect();
|
||||
const ratio = Math.max(0, Math.min(1, (clientX - rect.left) / rect.width));
|
||||
return ratio * audioDuration;
|
||||
}, [audioDuration]);
|
||||
|
||||
const handleInsertPointerDown = useCallback((
|
||||
id: string,
|
||||
e: React.PointerEvent
|
||||
) => {
|
||||
e.preventDefault();
|
||||
e.dataTransfer.dropEffect = "move";
|
||||
setDragOverIdx(idx);
|
||||
}, []);
|
||||
e.stopPropagation();
|
||||
setDragId(id);
|
||||
dragStartXRef.current = e.clientX;
|
||||
dragMovedRef.current = false;
|
||||
const ins = inserts.find((i) => i.id === id);
|
||||
dragStartValRef.current = ins?.start ?? 0;
|
||||
(e.target as HTMLElement).setPointerCapture(e.pointerId);
|
||||
}, [inserts]);
|
||||
|
||||
const handleDragLeave = useCallback(() => {
|
||||
setDragOverIdx(null);
|
||||
}, []);
|
||||
|
||||
const handleDrop = useCallback((toIdx: number, e: React.DragEvent) => {
|
||||
e.preventDefault();
|
||||
const fromIdx = parseInt(e.dataTransfer.getData("text/plain"), 10);
|
||||
if (!isNaN(fromIdx) && fromIdx !== toIdx) {
|
||||
onReorderSegment(fromIdx, toIdx);
|
||||
const handlePointerMove = useCallback((e: React.PointerEvent) => {
|
||||
if (!dragId) return;
|
||||
if (!dragMovedRef.current) {
|
||||
const dx = Math.abs(e.clientX - dragStartXRef.current);
|
||||
if (dx < DRAG_THRESHOLD) return;
|
||||
dragMovedRef.current = true;
|
||||
}
|
||||
setDragFromIdx(null);
|
||||
setDragOverIdx(null);
|
||||
}, [onReorderSegment]);
|
||||
const currentTime = getTimeFromClientX(e.clientX);
|
||||
const startTime = getTimeFromClientX(dragStartXRef.current);
|
||||
onMoveInsert(dragId, dragStartValRef.current + (currentTime - startTime));
|
||||
}, [dragId, getTimeFromClientX, onMoveInsert]);
|
||||
|
||||
const handleDragEnd = useCallback(() => {
|
||||
setDragFromIdx(null);
|
||||
setDragOverIdx(null);
|
||||
const handlePointerUp = useCallback(() => {
|
||||
setDragId(null);
|
||||
}, []);
|
||||
|
||||
// Filter visible vs overflow segments
|
||||
const visibleSegments = useMemo(() => segments.filter((s) => s.start < audioDuration), [segments, audioDuration]);
|
||||
const overflowSegments = useMemo(() => segments.filter((s) => s.start >= audioDuration), [segments, audioDuration]);
|
||||
const hasSegments = visibleSegments.length > 0;
|
||||
|
||||
const content = (
|
||||
<>
|
||||
<div className="flex items-center justify-between mb-3">
|
||||
@@ -239,109 +267,149 @@ export function TimelineEditor({
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Waveform — always rendered so ref stays mounted */}
|
||||
{/* Waveform */}
|
||||
<div className="relative mb-1">
|
||||
<div ref={waveCallbackRef} className="rounded-lg overflow-hidden bg-black/20 cursor-pointer" style={{ minHeight: 56 }} />
|
||||
</div>
|
||||
|
||||
{/* Segment blocks or empty placeholder */}
|
||||
{hasSegments ? (
|
||||
{/* Timeline visualization */}
|
||||
{hasPrimary && audioDuration > 0 ? (
|
||||
<>
|
||||
<div className="relative h-14 flex select-none">
|
||||
{/* Playhead — syncs with audio playback */}
|
||||
<div
|
||||
ref={timelineBarRef}
|
||||
className="relative select-none touch-none"
|
||||
style={{ minHeight: isMultiCam ? 80 : 56 }}
|
||||
onPointerMove={handlePointerMove}
|
||||
onPointerUp={handlePointerUp}
|
||||
onPointerLeave={handlePointerUp}
|
||||
>
|
||||
{/* Playhead */}
|
||||
<div
|
||||
ref={playheadRef}
|
||||
className="absolute top-0 h-full w-0.5 bg-fuchsia-400 z-10 pointer-events-none"
|
||||
className="absolute top-0 h-full w-0.5 bg-fuchsia-400 z-20 pointer-events-none"
|
||||
style={{ display: "none", left: "0%" }}
|
||||
/>
|
||||
{visibleSegments.map((seg, i) => {
|
||||
const left = (seg.start / audioDuration) * 100;
|
||||
const width = ((seg.end - seg.start) / audioDuration) * 100;
|
||||
const segDur = seg.end - seg.start;
|
||||
const isDragTarget = dragOverIdx === i && dragFromIdx !== i;
|
||||
|
||||
// Compute loop portion for the last visible segment
|
||||
const isLastVisible = i === visibleSegments.length - 1;
|
||||
let loopPercent = 0;
|
||||
if (isLastVisible && audioDuration > 0) {
|
||||
const mat = materials.find((m) => m.id === seg.materialId);
|
||||
const matDur = mat?.duration_sec ?? 0;
|
||||
const effDur = (seg.sourceEnd > seg.sourceStart)
|
||||
? (seg.sourceEnd - seg.sourceStart)
|
||||
: Math.max(matDur - seg.sourceStart, 0);
|
||||
if (effDur > 0 && segDur > effDur + 0.1) {
|
||||
loopPercent = ((segDur - effDur) / segDur) * 100;
|
||||
}
|
||||
}
|
||||
{/* Primary material background bar */}
|
||||
<button
|
||||
onClick={onClickPrimary}
|
||||
className="absolute inset-0 rounded-lg overflow-hidden border border-purple-500/30 hover:border-purple-500/50 transition-colors cursor-pointer"
|
||||
style={{ backgroundColor: "#8b5cf620" }}
|
||||
title={`主素材: ${primaryMaterial?.scene || primaryMaterial?.name || ""}${
|
||||
loopCount > 1 ? ` (${primaryEffective.toFixed(1)}s ×${loopCount.toFixed(1)} 循环)` : ""
|
||||
}\n点击设置截取范围`}
|
||||
>
|
||||
{/* Loop stripe pattern */}
|
||||
{loopCount > 1 && (
|
||||
<div
|
||||
className="absolute inset-0 pointer-events-none"
|
||||
style={{
|
||||
background: `repeating-linear-gradient(-45deg, transparent, transparent 6px, rgba(139,92,246,0.06) 6px, rgba(139,92,246,0.06) 12px)`,
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
<div className="absolute inset-0 flex items-center px-3">
|
||||
<span className="text-[11px] text-purple-300/80 truncate">
|
||||
主素材: {primaryMaterial?.scene || primaryMaterial?.name || ""}
|
||||
{loopCount > 1 && (
|
||||
<span className="text-purple-400/60 ml-1">
|
||||
({primaryEffective.toFixed(1)}s ×{loopCount.toFixed(1)} 循环)
|
||||
</span>
|
||||
)}
|
||||
{primarySourceStart > 0 && (
|
||||
<span className="text-amber-400/80 ml-1">✂ {primarySourceStart.toFixed(1)}s</span>
|
||||
)}
|
||||
</span>
|
||||
</div>
|
||||
</button>
|
||||
|
||||
{/* Insert blocks floating above primary */}
|
||||
{inserts.map((ins) => {
|
||||
const left = (ins.start / audioDuration) * 100;
|
||||
const width = ((ins.end - ins.start) / audioDuration) * 100;
|
||||
const insDur = ins.end - ins.start;
|
||||
const isDragging = dragId === ins.id;
|
||||
|
||||
return (
|
||||
<div key={seg.id} className="absolute top-0 h-full" style={{ left: `${left}%`, width: `${width}%` }}>
|
||||
<div
|
||||
key={ins.id}
|
||||
className={`absolute group min-h-[40px] ${isDragging ? "z-30" : "z-10"}`}
|
||||
style={{
|
||||
left: `${left}%`,
|
||||
width: `${width}%`,
|
||||
top: isMultiCam ? 12 : 4,
|
||||
bottom: isMultiCam ? 12 : 4,
|
||||
}}
|
||||
>
|
||||
{/* Main block body — move on drag, click opens ClipTrimmer */}
|
||||
<button
|
||||
draggable
|
||||
onDragStart={(e) => handleDragStart(i, e)}
|
||||
onDragOver={(e) => handleDragOver(i, e)}
|
||||
onDragLeave={handleDragLeave}
|
||||
onDrop={(e) => handleDrop(i, e)}
|
||||
onDragEnd={handleDragEnd}
|
||||
onClick={() => onClickSegment(seg)}
|
||||
className={`relative w-full h-full rounded-lg flex flex-col items-center justify-center overflow-hidden cursor-grab active:cursor-grabbing transition-all border ${
|
||||
isDragTarget
|
||||
? "ring-2 ring-purple-400 border-purple-400 scale-[1.02]"
|
||||
: dragFromIdx === i
|
||||
? "opacity-50 border-white/10"
|
||||
: "hover:opacity-90 border-white/10"
|
||||
className={`w-full h-full rounded-lg flex flex-col items-center justify-center overflow-hidden cursor-grab active:cursor-grabbing transition-all border ${
|
||||
isDragging
|
||||
? "ring-2 ring-white/40 scale-[1.02]"
|
||||
: "hover:brightness-110"
|
||||
}`}
|
||||
style={{ backgroundColor: seg.color + "33", borderColor: isDragTarget ? undefined : seg.color + "66" }}
|
||||
title={`拖拽可调换顺序 · 点击设置截取范围\n${seg.materialName}\n${segDur.toFixed(1)}s${loopPercent > 0 ? ` (含循环 ${(segDur * loopPercent / 100).toFixed(1)}s)` : ""}`}
|
||||
style={{
|
||||
backgroundColor: ins.color + "55",
|
||||
borderColor: ins.color + "88",
|
||||
}}
|
||||
onPointerDown={(e) => handleInsertPointerDown(ins.id, e)}
|
||||
onClick={() => {
|
||||
if (!dragMovedRef.current) onClickInsert(ins);
|
||||
}}
|
||||
title={`${ins.materialName} ${insDur.toFixed(1)}s\n点击设置截取范围`}
|
||||
>
|
||||
<GripVertical className="absolute top-0.5 left-0.5 h-3 w-3 text-white/30 z-[1]" />
|
||||
<span className="text-[11px] text-white/90 truncate max-w-full px-1 leading-tight z-[1]">
|
||||
{seg.materialName}
|
||||
{ins.materialName}
|
||||
</span>
|
||||
<span className="text-[10px] text-white/60 leading-tight z-[1]">
|
||||
{segDur.toFixed(1)}s
|
||||
{insDur.toFixed(1)}s
|
||||
</span>
|
||||
{seg.sourceStart > 0 && (
|
||||
{ins.sourceStart > 0 && (
|
||||
<span className="text-[9px] text-amber-400/80 leading-tight z-[1]">
|
||||
✂ {seg.sourceStart.toFixed(1)}s
|
||||
✂ {ins.sourceStart.toFixed(1)}s
|
||||
</span>
|
||||
)}
|
||||
{/* Loop fill stripe overlay */}
|
||||
{loopPercent > 0 && (
|
||||
<div
|
||||
className="absolute top-0 right-0 h-full pointer-events-none flex items-center justify-center"
|
||||
style={{
|
||||
width: `${loopPercent}%`,
|
||||
background: `repeating-linear-gradient(-45deg, transparent, transparent 3px, rgba(255,255,255,0.07) 3px, rgba(255,255,255,0.07) 6px)`,
|
||||
borderLeft: "1px dashed rgba(255,255,255,0.25)",
|
||||
}}
|
||||
>
|
||||
<span className="text-[9px] text-white/30">循环</span>
|
||||
</div>
|
||||
)}
|
||||
</button>
|
||||
|
||||
{/* Delete button */}
|
||||
<button
|
||||
className="absolute -top-1.5 -right-1.5 w-5 h-5 rounded-full bg-red-500/80 hover:bg-red-500 flex items-center justify-center opacity-40 sm:opacity-0 sm:group-hover:opacity-100 transition-opacity z-20"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
onRemoveInsert(ins.id);
|
||||
}}
|
||||
title="删除此插入"
|
||||
>
|
||||
<X className="w-3 h-3 text-white" />
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
|
||||
{/* Overflow segments — shown as gray chips */}
|
||||
{overflowSegments.length > 0 && (
|
||||
<div className="flex flex-wrap items-center gap-1.5 mt-1.5">
|
||||
<span className="text-[10px] text-gray-500">未使用:</span>
|
||||
{overflowSegments.map((seg) => (
|
||||
<span
|
||||
key={seg.id}
|
||||
className="text-[10px] text-gray-500 bg-white/5 border border-white/10 rounded px-1.5 py-0.5"
|
||||
{/* Insert candidates bar (multi-cam only) */}
|
||||
{isMultiCam && insertCandidates.length > 0 && (
|
||||
<div className="flex flex-wrap items-center gap-1.5 mt-2">
|
||||
<span className="text-[10px] text-gray-500">可插入:</span>
|
||||
{insertCandidates.map((mat) => (
|
||||
<button
|
||||
key={mat.id}
|
||||
className="flex items-center gap-0.5 text-[10px] text-gray-300 bg-white/5 border border-white/10 hover:border-white/30 rounded px-1.5 py-0.5 transition-colors"
|
||||
onClick={() => onAddInsert(mat.id)}
|
||||
title={`添加 "${mat.scene || mat.name}" 到时间轴`}
|
||||
>
|
||||
{seg.materialName}
|
||||
</span>
|
||||
<Plus className="w-2.5 h-2.5" />
|
||||
{mat.scene || mat.name}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<p className="text-[10px] text-gray-500 mt-1.5">
|
||||
点击波形定位播放 · 拖拽色块调换顺序 · 点击色块设置截取范围
|
||||
{isMultiCam
|
||||
? "点击主素材设置截取范围 · 拖拽插入块调整位置 · 点击插入块设置截取/时长"
|
||||
: "点击波形定位播放 · 点击素材条设置截取范围"
|
||||
}
|
||||
</p>
|
||||
</>
|
||||
) : (
|
||||
|
||||
@@ -0,0 +1,239 @@
|
||||
import { useCallback, useEffect, useState } from "react";
|
||||
import api from "@/shared/api/axios";
|
||||
import { ApiResponse, unwrap } from "@/shared/api/types";
|
||||
import { toast } from "sonner";
|
||||
|
||||
export type ScriptLearningStep = "input" | "analyzing" | "topics" | "generating" | "result";
|
||||
|
||||
const WORD_COUNT_MIN = 80;
|
||||
const WORD_COUNT_MAX = 1000;
|
||||
const DEFAULT_WORD_COUNT = "300";
|
||||
|
||||
interface UseScriptLearningOptions {
|
||||
isOpen: boolean;
|
||||
}
|
||||
|
||||
interface AnalyzeCreatorPayload {
|
||||
topics: string[];
|
||||
analysis_id: string;
|
||||
fetched_count: number;
|
||||
}
|
||||
|
||||
interface GenerateTopicScriptPayload {
|
||||
script: string;
|
||||
}
|
||||
|
||||
export const useScriptLearning = ({ isOpen }: UseScriptLearningOptions) => {
|
||||
const [step, setStep] = useState<ScriptLearningStep>("input");
|
||||
const [inputUrl, setInputUrl] = useState("");
|
||||
const [topics, setTopics] = useState<string[]>([]);
|
||||
const [selectedTopic, setSelectedTopic] = useState<string | null>(null);
|
||||
const [wordCount, setWordCount] = useState(DEFAULT_WORD_COUNT);
|
||||
const [generatedScript, setGeneratedScript] = useState("");
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [analysisId, setAnalysisId] = useState<string | null>(null);
|
||||
const [fetchedCount, setFetchedCount] = useState(0);
|
||||
|
||||
const resetAll = useCallback(() => {
|
||||
setStep("input");
|
||||
setInputUrl("");
|
||||
setTopics([]);
|
||||
setSelectedTopic(null);
|
||||
setWordCount(DEFAULT_WORD_COUNT);
|
||||
setGeneratedScript("");
|
||||
setError(null);
|
||||
setAnalysisId(null);
|
||||
setFetchedCount(0);
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
if (isOpen) {
|
||||
resetAll();
|
||||
}
|
||||
}, [isOpen, resetAll]);
|
||||
|
||||
const parseWordCount = useCallback((value: string): number | null => {
|
||||
const num = Number(value);
|
||||
if (!Number.isInteger(num)) {
|
||||
return null;
|
||||
}
|
||||
if (num < WORD_COUNT_MIN || num > WORD_COUNT_MAX) {
|
||||
return null;
|
||||
}
|
||||
return num;
|
||||
}, []);
|
||||
|
||||
const handleAnalyze = useCallback(async () => {
|
||||
const urlValue = inputUrl.trim();
|
||||
if (!urlValue) {
|
||||
setError("请先输入博主主页链接");
|
||||
return;
|
||||
}
|
||||
|
||||
setError(null);
|
||||
setStep("analyzing");
|
||||
|
||||
try {
|
||||
const { data: res } = await api.post<ApiResponse<AnalyzeCreatorPayload>>(
|
||||
"/api/tools/analyze-creator",
|
||||
{ url: urlValue },
|
||||
{ timeout: 60000 }
|
||||
);
|
||||
const payload = unwrap(res);
|
||||
const topicList = payload.topics || [];
|
||||
|
||||
if (topicList.length === 0) {
|
||||
throw new Error("未识别到可用话题，请更换链接重试");
|
||||
}
|
||||
|
||||
setTopics(topicList);
|
||||
setSelectedTopic(topicList[0]);
|
||||
setAnalysisId(payload.analysis_id || null);
|
||||
setFetchedCount(payload.fetched_count || 0);
|
||||
setGeneratedScript("");
|
||||
setStep("topics");
|
||||
} catch (err: unknown) {
|
||||
const axiosErr = err as {
|
||||
response?: { data?: { message?: string } };
|
||||
message?: string;
|
||||
};
|
||||
const msg = axiosErr.response?.data?.message || axiosErr.message || "分析失败，请稍后重试";
|
||||
setError(msg);
|
||||
setStep("input");
|
||||
}
|
||||
}, [inputUrl]);
|
||||
|
||||
const handleGenerate = useCallback(async () => {
|
||||
if (!analysisId) {
|
||||
setError("分析结果已失效，请重新分析");
|
||||
setStep("input");
|
||||
return;
|
||||
}
|
||||
if (!selectedTopic) {
|
||||
setError("请先选择一个话题");
|
||||
return;
|
||||
}
|
||||
|
||||
const count = parseWordCount(wordCount.trim());
|
||||
if (count === null) {
|
||||
setError(`目标字数需在 ${WORD_COUNT_MIN}-${WORD_COUNT_MAX} 之间`);
|
||||
return;
|
||||
}
|
||||
|
||||
setError(null);
|
||||
setStep("generating");
|
||||
|
||||
try {
|
||||
const { data: res } = await api.post<ApiResponse<GenerateTopicScriptPayload>>(
|
||||
"/api/tools/generate-topic-script",
|
||||
{
|
||||
analysis_id: analysisId,
|
||||
topic: selectedTopic,
|
||||
word_count: count,
|
||||
},
|
||||
{ timeout: 90000 }
|
||||
);
|
||||
const payload = unwrap(res);
|
||||
|
||||
const script = (payload.script || "").trim();
|
||||
if (!script) {
|
||||
throw new Error("生成内容为空，请重试");
|
||||
}
|
||||
|
||||
setGeneratedScript(script);
|
||||
setStep("result");
|
||||
} catch (err: unknown) {
|
||||
const axiosErr = err as {
|
||||
response?: { data?: { message?: string } };
|
||||
message?: string;
|
||||
code?: string;
|
||||
};
|
||||
let msg = axiosErr.response?.data?.message || axiosErr.message || "生成失败，请稍后重试";
|
||||
if (axiosErr.code === "ECONNABORTED" || /timeout/i.test(axiosErr.message || "")) {
|
||||
msg = "生成超时，请稍后重试（可适当减少目标字数）";
|
||||
}
|
||||
setError(msg);
|
||||
setStep("topics");
|
||||
}
|
||||
}, [analysisId, parseWordCount, selectedTopic, wordCount]);
|
||||
|
||||
const handleRegenerate = useCallback(async () => {
|
||||
await handleGenerate();
|
||||
}, [handleGenerate]);
|
||||
|
||||
const backToInput = useCallback(() => {
|
||||
setError(null);
|
||||
setStep("input");
|
||||
}, []);
|
||||
|
||||
const backToTopics = useCallback(() => {
|
||||
setError(null);
|
||||
setStep("topics");
|
||||
}, []);
|
||||
|
||||
const fallbackCopyTextToClipboard = useCallback((text: string) => {
|
||||
const textArea = document.createElement("textarea");
|
||||
textArea.value = text;
|
||||
textArea.style.top = "0";
|
||||
textArea.style.left = "0";
|
||||
textArea.style.position = "fixed";
|
||||
textArea.style.opacity = "0";
|
||||
|
||||
document.body.appendChild(textArea);
|
||||
textArea.focus();
|
||||
textArea.select();
|
||||
|
||||
try {
|
||||
const successful = document.execCommand("copy");
|
||||
if (successful) {
|
||||
toast.success("已复制到剪贴板");
|
||||
} else {
|
||||
toast.error("复制失败，请手动复制");
|
||||
}
|
||||
} catch {
|
||||
toast.error("复制失败，请手动复制");
|
||||
}
|
||||
|
||||
document.body.removeChild(textArea);
|
||||
}, []);
|
||||
|
||||
const copyToClipboard = useCallback(
|
||||
(text: string) => {
|
||||
if (navigator.clipboard && window.isSecureContext) {
|
||||
navigator.clipboard
|
||||
.writeText(text)
|
||||
.then(() => {
|
||||
toast.success("已复制到剪贴板");
|
||||
})
|
||||
.catch(() => {
|
||||
fallbackCopyTextToClipboard(text);
|
||||
});
|
||||
} else {
|
||||
fallbackCopyTextToClipboard(text);
|
||||
}
|
||||
},
|
||||
[fallbackCopyTextToClipboard]
|
||||
);
|
||||
|
||||
return {
|
||||
step,
|
||||
inputUrl,
|
||||
setInputUrl,
|
||||
topics,
|
||||
selectedTopic,
|
||||
setSelectedTopic,
|
||||
wordCount,
|
||||
setWordCount,
|
||||
generatedScript,
|
||||
error,
|
||||
analysisId,
|
||||
fetchedCount,
|
||||
handleAnalyze,
|
||||
handleGenerate,
|
||||
handleRegenerate,
|
||||
backToInput,
|
||||
backToTopics,
|
||||
resetAll,
|
||||
copyToClipboard,
|
||||
};
|
||||
};
|
||||
10
frontend/src/shared/types/timeline.ts
Normal file
10
frontend/src/shared/types/timeline.ts
Normal file
@@ -0,0 +1,10 @@
|
||||
export interface InsertSegment {
|
||||
id: string;
|
||||
materialId: string;
|
||||
materialName: string;
|
||||
start: number;
|
||||
end: number;
|
||||
sourceStart: number;
|
||||
sourceEnd: number;
|
||||
color: string;
|
||||
}
|
||||
@@ -11,6 +11,7 @@ interface AppModalProps {
|
||||
zIndexClassName?: string;
|
||||
panelClassName?: string;
|
||||
closeOnOverlay?: boolean;
|
||||
closeOnEsc?: boolean;
|
||||
lockBodyScroll?: boolean;
|
||||
}
|
||||
|
||||
@@ -21,6 +22,7 @@ export function AppModal({
|
||||
zIndexClassName = "z-[220]",
|
||||
panelClassName = "w-full max-w-2xl rounded-2xl border border-white/10 bg-[#171821]/95 shadow-[0_24px_80px_rgba(0,0,0,0.55)] overflow-hidden",
|
||||
closeOnOverlay = true,
|
||||
closeOnEsc = true,
|
||||
lockBodyScroll = true,
|
||||
}: AppModalProps) {
|
||||
const containerRef = useRef<HTMLDivElement | null>(null);
|
||||
@@ -34,7 +36,7 @@ export function AppModal({
|
||||
if (!isOpen) return;
|
||||
|
||||
const handleEsc = (event: KeyboardEvent) => {
|
||||
if (event.key === "Escape") onCloseRef.current();
|
||||
if (closeOnEsc && event.key === "Escape") onCloseRef.current();
|
||||
};
|
||||
|
||||
const previousActiveElement = document.activeElement as HTMLElement | null;
|
||||
@@ -69,7 +71,7 @@ export function AppModal({
|
||||
|
||||
previousActiveElement?.focus?.();
|
||||
};
|
||||
}, [isOpen, lockBodyScroll]);
|
||||
}, [closeOnEsc, isOpen, lockBodyScroll]);
|
||||
|
||||
if (!isOpen || typeof document === "undefined") return null;
|
||||
|
||||
|
||||
Reference in New Issue
Block a user