156 lines
4.2 KiB
Markdown
156 lines
4.2 KiB
Markdown
# MuseTalk 部署指南
|
||
|
||
## 硬件要求
|
||
|
||
| 配置 | 最低要求 | 推荐配置 |
|
||
|------|----------|----------|
|
||
| GPU | 8GB VRAM (如 RTX 3060) | 24GB VRAM (如 RTX 3090) |
|
||
| 内存 | 32GB | 64GB |
|
||
| CUDA | 11.7+ | 12.0+ |
|
||
|
||
---
|
||
|
||
## 📦 安装步骤
|
||
|
||
### 1. 克隆 MuseTalk 仓库
|
||
|
||
```bash
|
||
# 进入 ViGent 项目的 models 目录
|
||
cd /home/rongye/ProgramFiles/ViGent/models
|
||
|
||
# 克隆 MuseTalk 仓库
|
||
git clone https://github.com/TMElyralab/MuseTalk.git MuseTalk_repo
|
||
|
||
# 保留我们的自定义文件 (如果有)
|
||
# cp MuseTalk/DEPLOY.md MuseTalk_repo/
|
||
# cp MuseTalk/musetalk_api.py MuseTalk_repo/
|
||
|
||
# 替换目录
|
||
rm -rf MuseTalk
|
||
mv MuseTalk_repo MuseTalk
|
||
```
|
||
|
||
### 2. 创建虚拟环境
|
||
|
||
```bash
|
||
cd /home/rongye/ProgramFiles/ViGent/models/MuseTalk
|
||
conda create -n musetalk python=3.10 -y
|
||
conda activate musetalk
|
||
```
|
||
|
||
### 3. 安装 PyTorch (稳定版 2.0.1)
|
||
|
||
⚠️ **注意**：必须使用 PyTorch 2.0.1 配合 CUDA 11.8，因为这是 mmcv 预编译包支持的版本。
|
||
|
||
```bash
|
||
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
|
||
```
|
||
|
||
### 4. 安装 MuseTalk 依赖 (MMLab)
|
||
|
||
严格按照以下顺序和版本安装：
|
||
|
||
```bash
|
||
pip install -r requirements.txt
|
||
|
||
# 安装 mmlab 系列
|
||
pip install --no-cache-dir -U openmim
|
||
mim install mmengine
|
||
mim install "mmcv==2.0.1"
|
||
mim install "mmdet==3.1.0"
|
||
pip install chumpy --no-build-isolation
|
||
pip install "mmpose==1.1.0" --no-deps
|
||
```
|
||
|
||
### 5. 下载模型权重 ⬇️
|
||
|
||
> **注意**：模型目录结构极其重要，必须严格按照以下步骤操作。
|
||
|
||
```bash
|
||
cd /home/rongye/ProgramFiles/ViGent/models/MuseTalk/models
|
||
|
||
# 1. 准备目录
|
||
mkdir -p musetalk musetalkV15 dwpose syncnet face-parse-bisent
|
||
|
||
# 2. 从 HuggingFace 下载基础模型
|
||
pip install huggingface_hub
|
||
|
||
# MuseTalk v1
|
||
huggingface-cli download TMElyralab/MuseTalk --local-dir ./musetalk_tmp --include "*.pth" "*.json" "*.bin"
|
||
mv musetalk_tmp/* musetalk/ && rm -rf musetalk_tmp
|
||
|
||
# MuseTalk v1.5
|
||
huggingface-cli download TMElyralab/MuseTalk --local-dir ./musetalkV15_tmp --include "unet.pth"
|
||
mv musetalkV15_tmp/* musetalkV15/ && rm -rf musetalkV15_tmp
|
||
|
||
# SD-VAE
|
||
huggingface-cli download stabilityai/sd-vae-ft-mse --local-dir ./sd-vae-ft-mse
|
||
|
||
# Whisper
|
||
huggingface-cli download openai/whisper-tiny --local-dir ./whisper
|
||
|
||
# DWPose
|
||
huggingface-cli download yzd-v/DWPose dw-ll_ucoco_384.pth --local-dir ./dwpose
|
||
|
||
# SyncNet
|
||
huggingface-cli download ByteDance/LatentSync latentsync_syncnet.pt --local-dir ./syncnet
|
||
|
||
# 3. 下载 Face Parse 模型 (需从 Google Drive 或 PyTorch 官网)
|
||
cd face-parse-bisent
|
||
wget https://download.pytorch.org/models/resnet18-5c106cde.pth -O resnet18-5c106cde.pth
|
||
# 79999_iter.pth主要从Google Drive下载，或手动上传
|
||
# pip install gdown && gdown 154JgKpzCPW82qINcVieuPH3fZ2e0P812 -O 79999_iter.pth
|
||
cd ..
|
||
|
||
# 4. === 关键修复步骤 ===
|
||
# 创建必要的符号链接以匹配代码路径
|
||
ln -s sd-vae-ft-mse sd-vae
|
||
cd musetalk && ln -s musetalk.json config.json && cd ..
|
||
```
|
||
|
||
---
|
||
|
||
## 📂 最终目录结构验证
|
||
|
||
确保 `models/MuseTalk/models` 目录如下所示：
|
||
|
||
```
|
||
models/
|
||
├── musetalk/
|
||
│ ├── config.json -> musetalk.json # ⚠️ 必须有此软链
|
||
│ ├── musetalk.json
|
||
│ └── pytorch_model.bin
|
||
├── musetalkV15/
|
||
│ ├── musetalk.json
|
||
│ └── unet.pth
|
||
├── sd-vae -> sd-vae-ft-mse # ⚠️ 必须有此软链
|
||
├── sd-vae-ft-mse/
|
||
│ └── diffusion_pytorch_model.bin
|
||
├── whisper/
|
||
│ └── pytorch_model.bin ...
|
||
├── dwpose/
|
||
│ └── dw-ll_ucoco_384.pth
|
||
├── syncnet/
|
||
│ └── latentsync_syncnet.pt
|
||
└── face-parse-bisent/
|
||
├── 79999_iter.pth
|
||
└── resnet18-5c106cde.pth
|
||
```
|
||
|
||
---
|
||
|
||
## 🔧 验证安装
|
||
|
||
```bash
|
||
cd /home/rongye/ProgramFiles/ViGent/models/MuseTalk
|
||
conda activate musetalk
|
||
|
||
# 运行测试脚本 (需先准备测试素材)
|
||
# 确保 inference_config.yaml 格式正确
|
||
CUDA_VISIBLE_DEVICES=1 python -m scripts.inference \
|
||
--version v15 \
|
||
--inference_config configs/inference/test.yaml \
|
||
--result_dir ./results \
|
||
--use_float16
|
||
```
|