# MuseTalk 部署指南

## 硬件要求

| 配置 | 最低要求 | 推荐配置 |
|------|----------|----------|
| GPU | 8GB VRAM (如 RTX 3060) | 24GB VRAM (如 RTX 3090) |
| 内存 | 32GB | 64GB |
| CUDA | 11.7+ | 12.0+ |

---

## 📦 安装步骤

### 1. 克隆 MuseTalk 仓库

```bash
# 进入 ViGent 项目的 models 目录
cd /home/rongye/ProgramFiles/ViGent/models

# 克隆 MuseTalk 仓库
git clone https://github.com/TMElyralab/MuseTalk.git MuseTalk_repo

# 保留我们的自定义文件 (如果有)
# cp MuseTalk/DEPLOY.md MuseTalk_repo/
# cp MuseTalk/musetalk_api.py MuseTalk_repo/

# 替换目录
rm -rf MuseTalk
mv MuseTalk_repo MuseTalk
```

### 2. 创建虚拟环境

```bash
cd /home/rongye/ProgramFiles/ViGent/models/MuseTalk
conda create -n musetalk python=3.10 -y
conda activate musetalk
```

### 3. 安装 PyTorch (稳定版 2.0.1)

⚠️ **注意**：必须使用 PyTorch 2.0.1 配合 CUDA 11.8，因为这是 mmcv 预编译包支持的版本。

```bash
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
```

### 4. 安装 MuseTalk 依赖 (MMLab)

严格按照以下顺序和版本安装：

```bash
pip install -r requirements.txt

# 安装 mmlab 系列
pip install --no-cache-dir -U openmim
mim install mmengine
mim install "mmcv==2.0.1"
mim install "mmdet==3.1.0"
pip install chumpy --no-build-isolation
pip install "mmpose==1.1.0" --no-deps
```

### 5. 下载模型权重 ⬇️

> **注意**：模型目录结构极其重要，必须严格按照以下步骤操作。

```bash
cd /home/rongye/ProgramFiles/ViGent/models/MuseTalk/models

# 1. 准备目录
mkdir -p musetalk musetalkV15 dwpose syncnet face-parse-bisent

# 2. 从 HuggingFace 下载基础模型
pip install huggingface_hub

# MuseTalk v1
huggingface-cli download TMElyralab/MuseTalk --local-dir ./musetalk_tmp --include "*.pth" "*.json" "*.bin"
mv musetalk_tmp/* musetalk/ && rm -rf musetalk_tmp

# MuseTalk v1.5
huggingface-cli download TMElyralab/MuseTalk --local-dir ./musetalkV15_tmp --include "unet.pth"
mv musetalkV15_tmp/* musetalkV15/ && rm -rf musetalkV15_tmp

# SD-VAE
huggingface-cli download stabilityai/sd-vae-ft-mse --local-dir ./sd-vae-ft-mse

# Whisper
huggingface-cli download openai/whisper-tiny --local-dir ./whisper

# DWPose
huggingface-cli download yzd-v/DWPose dw-ll_ucoco_384.pth --local-dir ./dwpose

# SyncNet
huggingface-cli download ByteDance/LatentSync latentsync_syncnet.pt --local-dir ./syncnet

# 3. 下载 Face Parse 模型 (需从 Google Drive 或 PyTorch 官网)
cd face-parse-bisent
wget https://download.pytorch.org/models/resnet18-5c106cde.pth -O resnet18-5c106cde.pth
# 79999_iter.pth主要从Google Drive下载，或手动上传
# pip install gdown && gdown 154JgKpzCPW82qINcVieuPH3fZ2e0P812 -O 79999_iter.pth
cd ..

# 4. === 关键修复步骤 ===
# 创建必要的符号链接以匹配代码路径
ln -s sd-vae-ft-mse sd-vae
cd musetalk && ln -s musetalk.json config.json && cd ..
```

---

## 📂 最终目录结构验证

确保 `models/MuseTalk/models` 目录如下所示：

```
models/
├── musetalk/
│   ├── config.json -> musetalk.json  # ⚠️ 必须有此软链
│   ├── musetalk.json
│   └── pytorch_model.bin
├── musetalkV15/
│   ├── musetalk.json
│   └── unet.pth
├── sd-vae -> sd-vae-ft-mse           # ⚠️ 必须有此软链
├── sd-vae-ft-mse/
│   └── diffusion_pytorch_model.bin
├── whisper/
│   └── pytorch_model.bin ...
├── dwpose/
│   └── dw-ll_ucoco_384.pth
├── syncnet/
│   └── latentsync_syncnet.pt
└── face-parse-bisent/
    ├── 79999_iter.pth
    └── resnet18-5c106cde.pth
```

---

## 🔧 验证安装

```bash
cd /home/rongye/ProgramFiles/ViGent/models/MuseTalk
conda activate musetalk

# 运行测试脚本 (需先准备测试素材)
# 确保 inference_config.yaml 格式正确
CUDA_VISIBLE_DEVICES=1 python -m scripts.inference \
    --version v15 \
    --inference_config configs/inference/test.yaml \
    --result_dir ./results \
    --use_float16
```