文档重构

This commit is contained in:
Kevin Wong
2026-01-06 17:29:46 +08:00
parent 2ecfb39508
commit 13e261c2a8
5 changed files with 210 additions and 762 deletions

View File

@@ -68,23 +68,28 @@ NaviGlass/Docs/
> ⚠️ **仅当用户明确要求更新 `task_complete.md` 时才更新**
### 更新内容
```markdown
## ✅ 已完成任务
### 更新原则
- **格式一致性**:直接参考 `task_complete.md` 现有格式(列表/表格)追加内容。
- **进度更新**:仅在阶段性里程碑时更新进度百分比。
### [模块名称]
- [x] 功能描述 (Day N)
- [x] 子功能 1
```
### 🔍 完整性检查清单 (必做)
每次更新 `task_complete.md` 时,必须**逐一检查**以下所有板块:
### 进度表更新
```markdown
| 模块 | 完成度 | 状态 |
|------|--------|------|
| 网络通信 | 100% | ✅ 完成 |
```
1. **文件头部 & 导航**
- [ ] `更新时间`:必须是当天日期
- [ ] `整体进度`:简述当前状态 (e.g. "优化交付")
- [ ] `快速导航`Day 范围与文档一致
2. **核心任务区**
- [ ] `已完成任务`:添加新的 [x] 项目
- [ ] `后续规划与遗留问题`:管理三色板块 (优先/债务/未来),移除已完成项
3. **统计与回顾**
- [ ] `进度统计`:更新对应模块的状态描述和百分比
- [ ] `里程碑`:若有重大进展,追加 `## Milestone N` (注意用 **完成时间**)
4. **底部链接**
- [ ] `时间线`:追加今日概括,确保与 DayLog 一致
- [ ] `相关文档`:更新 DayLog 链接范围 (e.g. `Day1-Day28`)
- 仅在阶段性里程碑时更新进度百分比
> **口诀**:头尾时间要对齐,任务规划两手抓,里程碑上别落下。
---
@@ -106,4 +111,4 @@ NaviGlass/Docs/
---
**最后更新**2025-12-25
**最后更新**2026-01-06

60
DevLogs/Day28.md Normal file
View File

@@ -0,0 +1,60 @@
# Day 28: 室内导航优化与 Bug 修复 (2026-01-06)
## 🎯 今日目标
1. **修复室内导航无语音播报问题**
2. **优化“未检测到可行走区域”的播报逻辑,消除闪烁**
3. **切换到更精准的室内分割模型 (14类)**
4. **解决 TTS 音频末尾的“啪”声**
5. **排查终端 WiFi 断连后无法重连的问题**
## 🔧 实施细节
### 1. 模型切换与优化
- **切换模型**: 从原 `yolo11l-seg-indoor.engine` (22类) 切换为 `yolo11l-seg-indoor14.engine` (14类专为室内地板优化)。
- **类别映射更新**: 修改 `workflow_indoor.py` 以适配 14 类数据集 (MIT Indoor Subset)
- 可行走区域:`floor(0)`, `corridor(1)`, `sidewalk(2)`
- 静态障碍物:`chair(3)`, `table(4)`, `sofa_bed(5)`, `cabinet(11)`, `trash_can(12)`
- 兴趣点:`door(6)`, `elevator(7)`, `stairs(8)`
- 动态障碍:`person(10)`
- **效果**: 地板 (`floor`) 检测置信度显著提升0.85+),彻底解决了“无法识别地面”的问题。
- **识别逻辑优化**:
- **视觉持久化**: 引入可行走区域 Mask 缓冲机制,防止因单帧检测失败导致的“绿道闪烁”现象。
- **阈值微调**: 将 `WALKABLE_MIN_AREA` 降至 50提高对狭窄路段或远距离地面的检出灵敏度。
- **可视化增强**: 完善 `_draw_indoor_visualizations`,确保实时渲染分割掩膜与避障框。
### 2. 语音播报修复
- **强制首播机制**: 修复了 `IndoorNavigator` 中“未检测到可行走区域”的节流逻辑。
- **问题**: 初始化时间 `last_no_walkable_time` 为 0导致首次检测失败时由于时间差过大或逻辑判断失误被跳过。
- **修复**: 增加显式判断 `if self.last_no_walkable_time == 0`,确保首次检测到无路时**立即播报**,后续保持 8 秒间隔。
- **启动即时反馈**: 在 `app_main.py` 中,收到“室内导航”指令后,立即播报“**室内导航已启动,正在寻找可行走区域**”,消除用户等待时的不确定感。
- **崩溃修复**: 移除了新模型中不存在的 `CLASS_TOILET``CLASS_SINK` 引用,解决了后台报错导致的导航中断。
- **语音识别优化**:
- **别名扩展**: 针对 SenseVoice 的同音误识,增加了 `["四内导航", "思维导航", "失内导航", "室类导航"]` 等别名映射,确保“室内导航”指令的高唤醒率。
- **短指令支持**: 允许单独说“导航”触发盲道导航,防止吞字导致命令失效。
### 3. 音频体验优化
- **消除“啪”声**: 在 `app_main.py` 的 TTS 播放流末尾,将静音填充时长从 60ms 增加到 **200ms**,有效消除了音频截断带来的爆音。
### 4. 终端 WiFi 重连问题分析
- **现象**: 终端离开 WiFi 范围后断网,返回后无法自动重连。
- **分析**: 客户端程序 (`NaviGlassClient`) 的 WebSocket 连接逻辑已有无限重试和指数退避机制。无法重连是因为底层操作系统 (Tina Linux) 的 WiFi 接口在断开后未自动重新关联 AP。
- **结论**: 此为系统配置问题,非 App 代码 Bug。
- **建议**: 在终端部署后台脚本 (`wifi_check.sh`),定期检测网关连通性并在断网时重启 WiFi 接口。
## 📊 结果验证
-**地面检测**: 14类模型能稳定识别大面积 `floor` (area > 60000)。
-**语音反馈**: 启动导航和无路警告均能正常听到,不再静默。
-**稳定性**: 修复未定义变量错误后,导航进程稳定运行。
- ⚠️ **遗留问题**: 终端 WiFi 自动重连需在设备端通过脚本解决。
### 5. 文档体系重构与完善
- **系统架构图更新**: 重写 `implementation_plan_complete.md`,新增基于 Mermaid 的端云协同架构图,更新至 Day 28 状态。
- **任务清单优化**: 全面整理 `task_complete.md`,合并“技术债务/未来规划”板块,优化快速导航与排版。
- **技术文档刷新**: 更新《技术栈对比》与《服务器技术栈》,纳入室内导盲 (14类) 与新 AI 管道信息。
- **更新规则精简**: 优化 `DOC_RULES.md`,移除冗余示例,确立“轻量化检查清单”原则。
---
## 📝 下一步计划
1. 在真实环境中实测 14 类模型的避障效果。
2. (终端侧) 部署 WiFi 保活脚本。

View File

@@ -1,693 +1,105 @@
# Avaota F1 <20><><EFBFBD><EFBFBD><E887AC><EFBFBD><EFBFBD>
# NaviGlass 系统架构与实现详情
**<EFBFBD><EFBFBD>𧋦**嚗鯝3.5
**<EFBFBD><EFBFBD>**嚗?025-12-17
**撟喳蝱**嚗鋫vaota F1 (<28><EFBFBD> V821 / 32-bit RISC-V)
**憿寧𤌍餈𥕦漲**嚗?00% - [<EFBFBD><EFBFBD>隞餃𦛚皜<EFBFBD><EFBFBD>](task_complete.md)
**更新日期**: 2026-01-06 (Day 28)
**项目状态**: ✅ 核心功能完备 / 室内导盲优化交付
---
## <EFBFBD><EFBFBD> <20><>﹝霂湔<E99C82>
## 🏗️ 系统总体架构
<EFBFBD><EFBFBD><EFBFBD><EFBFBD>瘜其<EFBFBD>**<2A><><EFBFBD><EFBFBD><E887AC><EFBFBD><E59581><EFBFBD><EFBFBD><EFBFBD>𠗕<EFBFBD>寡圾<E5AFA1><EFBFBD>蝏誯<E89D8F><E8AAAF><EFBFBD>**<2A>?
<EFBFBD><EFBFBD><EFBFBD><EFBFBD>嚗?- <20><> **隞餃𦛚餈𥕦漲<F0A595A6><E6BCB2><EFBFBD>蝔讠<E89D94><E8AEA0><EFBFBD>𧒄<EFBFBD>渡瑪** <20>?`task_complete.md`
- <20><> **撘<><E69298>烐𠯫敹𡑒祕<F0A19192>?* <20>?`Day1-8.md`
NaviGlass 采用 **端云协同** 架构,将即时性要求高的采集/播放任务放在前端眼镜终端,将算力密集的 AI 推理与大模型交互放在高性能 GPU 服务器端。
---
```mermaid
graph TD
subgraph Client [Avaota F1 终端]
Cam[摄像头 GC2083] -->|JPEG 流| WS_Cam[WebSocket /ws/camera]
Mic[麦克风] -->|PCM 流| WS_Audio[WebSocket /ws_audio]
IMU[传感器 ICM-42688] -->|UDP JSON| UDP_IMU[UDP :12345]
WS_Audio -->|TTS PCM| Spk[扬声器]
end
## <20><20><><EFBFBD><EFBFBD><E88880><EFBFBD>
subgraph Server [RTX 3090 服务器]
WS_Cam --> |TurboJPEG| Visual[视觉感知中心]
WS_Audio --> |SenseVoice| ASR[语音识别]
UDP_IMU --> |Fusion| NavLogic[导航逻辑]
### 蝖砌辣憭𤥁挽
Visual --> |YOLOv11-seg| Lane[盲道检测]
Visual --> |YOLOv11-seg| Indoor[室内分割]
Visual --> |YOLOv11-det| Light[红绿灯检测]
| 璅<E79285> | <20>见噡/<2F>亙藁 | 摰䂿緵<E482BF><EFBFBD> | <20>喲睸<E596B2><E79DB8>㺭 |
|------|-----------|---------|----------|
| **<EFBFBD><EFBFBD>颲枏枂** | MAX98357A (I2S) | Device Tree + ALSA | PD12/PD13/PD15, 16kHz |
| **<EFBFBD><EFBFBD>颲枏<EFBFBD>** | <20>輯蝸璅⊥<E79285>暻血<E69ABB>憌?| Audio Codec ADC | MIC Gain=25, 16kHz Mono |
| **IMU** | ICM-42688-P | GPIO 璅⊥<E79285> SPI | 簣16g, 簣2000簞/s, ~500kHz |
| **<EFBFBD><EFBFBD><EFBFBD>憭?* | GC2083 (MIPI) | MPP 獢<>沲 | 1280x720@20fps, JPEG Q80 |
| **蝵𤑳<E89DB5>** | 隞亙云蝵?WiFi | <20>輯蝸 | - |
ASR --> |Text| LLM[GLM-4V-Flash]
LLM --> |Response| EdgeTTS[语音合成]
EdgeTTS --> |PCM| WS_Audio
### 頧臭辣<E887AD><EFBFBD>
Lane --> |Guidance| NavMaster[导航总控]
Indoor --> |Guidance| NavMaster
NavMaster --> |Prompt| EdgeTTS
end
| 蝏<>辣 | <20><><EFBFBD><EFBFBD> | 霂湔<E99C82> |
|------|--------|------|
| **AudioCapture** | ALSA API | 暻血<E69ABB>憌𤾸<E6868C><F0A4BEB8><EFBFBD>S16_LE<4C><EFBFBD> |
| **AudioPlayer** | ALSA API | <20>砍ㄟ<E7A08D>冽偘<E586BD>?|
| **ICM42688** | GPIO + SPI<50>讛悅 | <20><EFBFBD><E88AB8>訕PI撽勗𢆡 |
| **Camera** | MPP (VI/ISP/VENC) | JPEG蝖砌辣蝻𣇉<E89DBB> |
| **UDPSender** | BSD Socket | <20><EFBFBD><E8B9B1><EFBFBD>?|
| **WSClient** | <20><EFBFBD><E88AB8>?+ BSD Socket | WebSocket摰<E691B0>蝡荔<E89DA1><E88D94><EFBFBD>鈭諹<E988AD><E8ABB9>嗆㺭<E59786><EFBFBD> |
| **HTTPClient** | libcurl | HTTP摰<E691B0>蝡?|
### 蝻𤥁<E89DBB>撌亙<E6928C><E4BA99><EFBFBD><E6A998>湔鰵嚗帋蝙<E5B88B>?musl嚗?
```makefile
SDK_ROOT := ~/ProgramFiles/AvaotaF1/avaota_sdk/tina-v821-release
# <20>𩤃<EFBFBD> <20>喲睸靽格迤嚗帋蝙<E5B88B>?musl 撌亙<E6928C><E4BA99><EFBFBD>Day 9嚗?TOOLCHAIN_DIR := $(SDK_ROOT)/prebuilt/rootfsbuilt/riscv/nds32le-linux-musl-v5d/bin
CROSS_COMPILE := riscv32-linux-musl-
# <20><EFBFBD>蝵殷<E89DB5>銝滚<E98A9D>摰孵<E691B0><E5ADB5>烐踎嚗?# TOOLCHAIN_DIR := $(SDK_ROOT)/out/toolchain/nds32le-linux-glibc-v5d/bin
# CROSS_COMPILE := riscv32-unknown-linux-
CC := $(TOOLCHAIN_DIR)/$(CROSS_COMPILE)gcc
CXX := $(TOOLCHAIN_DIR)/$(CROSS_COMPILE)g++
```
**<EFBFBD>喲睸<EFBFBD>暹𦻖摨?*嚗?0+ <20><EFBFBD><E8B9B1><EFBFBD>嚗㚁<E59A97>
- MPP獢<50>沲嚗惨w_mpp, media_utils, awion
- ISP憭<50><E686AD>嚗?2銝杷SP摨?- 閫<><E996AB>蝻𤥁圾<F0A4A581><E59CBE><EFBFBD>vencoder, vdecoder, Cedar摨?- <20><EFBFBD><EFBFBD><E686AD>嚗惨decoder, aencoder, AGC, AEC
- <20><><EFBFBD><EFBFBD>嚗饢uxers, demuxer, parser
---
## <20><20><><EFBFBD>舫𠗕<E888AB><EFBFBD><EFBFBD><E996AB><EFBFBD><EFBFBD>
### 1. I2S <20><EFBFBD>颲枏枂<E69E8F>滨蔭
#### <20><EFBFBD>
- Device Tree <20><EFBFBD><E598A5>滨蔭<E6BBA8>躰秤
- 撘閗<E69298><E99697><EFBFBD><E8A098><EFBFBD>
- `simple-audio-card` <20><EFBFBD><EFBFBD>虜撌乩<E6928C>
#### 閫<><E996AB><EFBFBD><EFBFBD>
```dts
// <20><><EFBFBD>?LVDS <20>𣂼<EFBFBD><F0A382BC>滨蔭嚗䔶蛹瘥譍葵撘閗<E69298><E99697>𥕦遣<F0A595A6><EFBFBD><E7A589><EFBFBD><EFBFBD>
&pio {
sndcodec_pins_a: sndcodec@0 {
pins = "PD12";
function = "i2s0";
};
sndcodec_pins_b: sndcodec@1 {
pins = "PD13";
function = "i2s0";
};
sndcodec_pins_c: sndcodec@2 {
pins = "PD15";
function = "i2s0";
};
};
```
**<EFBFBD>喲睸蝏誯<EFBFBD>**嚗?- 雿輻鍂 `pins` + `function` <20><><EFBFBD>撅墧<E69285>?- <20><EFBFBD> `allwinner,pins` 蝑㕑䌊摰帋<E691B0>撅墧<E69285>?- 瘥譍葵撘閗<E69298><E99697><EFBFBD><E7A589><EFBFBD><EFBFBD><EFBFBD><EFBFBD><E6B994>?
---
### 2. ICM-42688 <20>帋縑<E5B88B><EFBFBD><E5AF9E><EFBFBD>
#### I2C <20><EFBFBD>撠肽<E692A0><EFBFBD>仃韐伐<E99F90><E4BC90>?
**<2A><EFBFBD>**嚗?- PD1 鋡怠<E98BA1>隞硋<E99A9E><E7A18B><EFBFBD><E8B3A2>?- PL2/PL3 銝齿𣈲<E9BDBF>?TWI0
- 霈曉<E99C88><E69B89><EFBFBD><E6BB9A><EFBFBD> (0x68) 雿<><E99BBF>蝏嘥<E89D8F>摮睃膥霂餃<E99C82>
**撠肽<E692A0><EFBFBD><E9A488><EFBFBD><EFBFBD>**嚗?1. GPIO 璅⊥<E79285> I2C
2. <20>湔揢<E6B994><EFBFBD>撘閗<E69298> (0x69)
3.<><EFBFBD><EFBFBD>撱嗉<E692B1>
4. Bank 撖<><E69296><EFBFBD><EFBFBD><E585B8>?
**蝏栞捏**嚗𡁜虾<F0A1819C><EFBFBD><EFBFBD>鸌畾𠰴<E795BE>憪见<E686AA>摨誩<E691A8>嚗峕𦆮撘<F0A686AE>2C<32><EFBFBD>
#### SPI <20><EFBFBD>摰䂿緵嚗<E7B7B5><E59A97><EFBFBD><EFBFBD><EFBFBD><EFBFBD>?
**摰䂿緵蝏<E7B7B5><E89D8F>**嚗?```cpp
// GPIO 璅⊥<E79285> SPI Mode 0 (CPOL=0, CPHA=0)
void spiTransfer(uint8_t data) {
for (int i = 7; i >= 0; i--) {
gpio_set_value(MOSI, (data >> i) & 0x01);
usleep(1); // ~500kHz
gpio_set_value(SCLK, 1); // 銝𠰴<E98A9D>瘝? usleep(1);
gpio_set_value(SCLK, 0);
}
}
```
**撘閗<E69298><E99697>滨蔭**嚗?- SCLK: PD3 (GPIO 99)
- MOSI: PD2 (GPIO 98)
- MISO: PD4 (GPIO 100)
- CS: PD5 (GPIO 101)
**<2A><EFBFBD>**嚗?- <20>笔漲嚗鰺500kHz
- WHO_AM_I 霂<><E99C82>嚗?x47 <20>?- <20>唳旿蝔喳<E89D94><E596B3><EFBFBD>隡条<E99AA1>
---
### 3. <20><EFBFBD><E8B9B1><EFBFBD>霂煾曎<E785BE>仿◇摨?
#### <20><EFBFBD>
```
undefined reference to `pthread_create`
```
#### 閫<><E996AB><EFBFBD><EFBFBD>
```makefile
# <20>躰秤憿箏<E686BF>
LDFLAGS += -lpthread -lssl -lcrypto
# 甇<>憿箏<E686BF>
LDFLAGS += -lssl -lcrypto -lpthread -lm -lstdc++
```
**閫<><E996AB>**嚗?1. 銝𡁜𦛚摨枏銁<E69E8F>?2. 蝟餌<E89D9F>摨枏銁<E69E8F>?3. `-lpthread` `-lm` `-lstdc++` <20><EFBFBD><E69AB9>?
---
### 4. MPP 獢<><EFBFBD><E6B2B2><EFBFBD>
#### 蝻枏<E89DBB><E69E8F><EFBFBD>蝵桐<E89DB5><E6A190>?
**<2A><EFBFBD>**嚗间BV 蝻枏<E89DBB><E69E8F>箸滯<E7AEB8>箏紡<E7AE8F>游葷銝
**閫<><E996AB><EFBFBD><EFBFBD>**嚗?```cpp
// VI 蝻枏<E89DBB><E69E8F><EFBFBD>蝵?vipp_attr.nbufs = 5; // 5銝杼I蝻枏<E89DBB><E69E8F>?vipp_attr.nplanes = 1;
// VBV <20>滨蔭
aw_enc_attr.VeAttr.mMaxKeyInterval = 30;
aw_enc_attr.VeAttr.mVbvBufferSize = 4 * 1024 * 1024; // 4MB
```
**<2A>喲睸蝏誯<E89D8F>**嚗?- VI蝻枏<E89DBB><E69E8F><EFBFBD>摰𡏭<E691B0>憭改<E686AD><E694B9><EFBFBD><EFBFBD><EFBFBD><EFBFBD>嚗?- VBV<42><56><EFBFBD>寞旿<E5AF9E><E697BF><EFBFBD><EFBFBD>
- JPEG蝻𣇉<E89DBB>韐券<E99F90>80銝箸<E98A9D>雿喳像銵∠<E98AB5>
---
### 5. 撌亙<E6928C><E4BA99><EFBFBD>蝵格𤣰<E6A0BC>?(Day 7-8)
#### <20><EFBFBD>
- README.md <20>𣂼<EFBFBD> `riscv32-linux-musl` 撌亙<E6928C><E4BA99><EFBFBD>摮睃銁
- 憭帋葵Makefile<6C>滨蔭銝滢<E98A9D><E6BBA2>?- <20><><EFBFBD>𡁏𧋦頝臬<E9A09D>瘛瑚僚
#### 閫<><E996AB><EFBFBD><E9A488>
1. <20>亥砭 `tina_files_clean.csv` <20><>辣蝝<E89D9D>
2. <20><EFBFBD><E586BD>膥銝𢠃<E98A9D><EFBFBD><E99C82><EFBFBD><EFBFBD>楝敺?3. <20>𤑳緵 musl 撌亙<E6928C><E4BA99>曆蛹<E69B86>讠憬<E8AEA0><E686AC><EFBFBD><EFBFBD>芾圾<E88ABE><EFBFBD>
4. 蝖株恕雿輻鍂 glibc 撌亙<E6928C><E4BA99><EFBFBD>撌脰圾<E884B0>见虾<E8A781><EFBFBD>
#### <20><><EFBFBD><E89D8F>蝵?```bash
# 摰鮋<E691B0>撌亙<E6928C><E4BA99>曇楝敺?out/toolchain/nds32le-linux-glibc-v5d/bin/
# 蝻𤥁<E89DBB><F0A4A581><EFBFBD>蝚血噡<E8A180>暹𦻖嚗?riscv32-unknown-linux-g++ -> riscv32-linux-g++
# C摨?glibc (<28>?musl)
Client <--> |FRP Tunnel| Server
```
---
### 6. Cedar 摨㯄曎<E3AF84>仿䔮憸?(Day 8)
## 🧠 核心模块实现详情
#### <20><EFBFBD>
```
undefined reference to `CDC_LOG_LEVEL_NAME'
undefined reference to `CDC_GLOBAL_LOG_LEVEL'
```
### 1. 视觉感知中心 (Visual Perception)
**文件**: `NaviGlassServer/navigation_master.py`, `workflow_*.py`
#### <20><EFBFBD>
銝餌<EFBFBD>摨?Makefile 蝻箏<E89DBB>摰峕㟲<E5B395>?Cedar 憭𡁜<E686AD>雿枏<E99BBF><E69E8F>暹𦻖
* **多模态导航状态机**:
* `IDLE`: 待机/闲聊模式
* `BLINDPATH_NAV`: 盲道导航 (Outdoor)
* `INDOOR_NAV`: 室内导盲 (Indoor)
* `TRAFFIC_LIGHT`: 红绿灯检测
* `ITEM_SEARCH`: 物品寻找
* **室内导航 (Indoor) [Day 28 New]**:
* **模型**: `yolo11l-seg-indoor14.engine` (TensorRT FP16)
* **策略**: 语义分割 (Seg) 优先。提取 `floor` (CLSID 0) 生成可行走区域掩膜 (Mask),计算中心点偏移量进行方向引导。
* **避障**: 检测 `chair`, `table`, `cabinet` 等障碍物,结合深度/位置给出避让提示。
* **盲道导航 (Outdoor)**:
* **模型**: `yolo11l-seg-blind.engine`
* **算法**: `FindBestPath` 算法,计算盲道分割掩膜的主方向,引导用户居中行走。
#### 閫<><E996AB><EFBFBD><EFBFBD>
瘛餃<EFBFBD> 60+ 銝芷<E98A9D><E88AB7><EFBFBD><EFBFBD>嚗?```makefile
# Cedar <20><EFBFBD>摨橒<E691A8><E6A992>喲睸嚗<E79DB8><E59A97>
LDFLAGS += -lcdc_base -lcdx_base
### 2. AI 交互管道 (Voice Interaction)
**文件**: `NaviGlassServer/app_main.py`
# <20><EFBFBD><EFBFBD><E686AD>
LDFLAGS += -ladecoder -lResample -lAudioVps -laac -lwav
LDFLAGS += -lcedarx_aencoder -laacenc -lAgc -lAec -lAns
* **ASR (语音识别)**: FunASR/SenseVoiceSmall (本地运行,强制中文 `zh`)。
* **LLM (大模型)**: ZhipuAI GLM-4.6v-Flash (通过 API)。支持视觉理解,用于 "这是什么"、"前方的文字" 等通用问答。
* **TTS (语音合成)**: EdgeTTS (本地/在线)。生成 16kHz PCM 音频流,并通过 WebSocket 实时推送到客户端播放。
# Muxer/Demuxer
LDFLAGS += -lmuxers -lcedarxdemuxer -lcdx_parser
LDFLAGS += -lmp4_muxer -lraw_muxer -lmpeg2ts_muxer
# 閫<><E996AB>蝻𤥁圾<F0A4A581>?LDFLAGS += -lvencoder -lvdecoder -lvideoengine -lawmjpegplus
# <20>滨蔭閫<E894AD><E996AB>
LDFLAGS += -lPluginMpp -lIniParserMpp -lsample_confparser
# <20>曄內摨?LDFLAGS += -lcedarxrender -lhwdisplay
```
### 3. 服务器性能优化 (Performance)
**关键技术**:
* **TensorRT 加速**: 所有 YOLO 模型导出为 `.engine`,启用 FP16 半精度。
* **TurboJPEG**: 替代 OpenCV 进行 JPEG 编解码,速度提升 3 倍。
* **零拷贝直传**: 非导航模式下,直接转发 JPEG 数据流给前端网页不进行解码CPU 占用降低 90%。
* **GPU 线程池**: `ThreadPoolExecutor` 管理 GPU 推理任务,避免 GIL 锁阻塞 Websocket 事件循环。
---
### 7. musl/glibc 撌亙<E6928C><E4BA99><EFBFBD>摰寞<E691B0>?(Day 9) <20>?
#### <20><EFBFBD>
<EFBFBD><EFBFBD>烐踎餈鞱<EFBFBD> `musl libc 1.2.4`嚗䔶<E59A97>蝔见<E89D94>雿輻鍂 `glibc` 蝻𤥁<E89DBB><EFBFBD><EFBFBD><EFBFBD>
- <20><EFBFBD><E586BD><EFBFBD>亙膥銝滚龪<E6BB9A>?- 蝚血噡銝滚<E98A9D>摰?(`__register_atfork` 蝑?
- 蝔见<E89D94><E8A781><EFBFBD>餈鞱<E9A488>
## 🕶️ 终端硬件与嵌入式实现
#### 閫<><E996AB><EFBFBD><EFBFBD>
```makefile
# 靽格迤撌亙<E6928C><E4BA99>曇楝敺<E6A59D><E695BA><EFBFBD>喲睸<E596B2>𤑳緵嚗饢usl <20>?prebuilt <20><EFBFBD> out/toolchain嚗?USE_MUSL := 1
**平台**: Avaota F1 (全志 V821 / RISC-V 架构)
**系统**: Tina Linux (OpenWrt base, musl libc)
ifeq ($(USE_MUSL),1)
TOOLCHAIN_DIR := $(SDK_ROOT)/prebuilt/rootfsbuilt/riscv/nds32le-linux-musl-v5d/bin
CROSS_COMPILE := riscv32-linux-musl-
endif
```
### 1. 传感器驱动
* **摄像头**:通过 MPP (Media Process Platform) 框架直接采集 ISP 处理后的图像,硬件编码为 JPEG。
* **IMU**: 使用 GPIO 模拟 SPI 驱动 ICM-42688实现 10Hz 数据采集。
* **音频**: ALSA 架构。
* 播放: I2S 接口 (hw:1,0) -> MAX98357A 功放。
* 录音: Audio Codec (hw:0,0) -> 驻极体麦克风。
**撉諹<E69289><E8ABB9><EFBFBD>**嚗?```bash
# 璉<><E79289>亙𢆡<E4BA99><F0A286A1><EFBFBD>亙膥
readelf -l avaota_client | grep interpreter
# 摨磰<E691A8><E7A3B0><EFBFBD>/lib32/ld.so.1
# 撘<><E69298>烐踎銝𠰴<E98A9D>撱箇泵<E7AE87>琿曎<E790BF>?ln -s /lib32/ilp32d/libc.so /lib32/ld.so.1
```
**蝏𤘪<E89D8F>**嚗尠<E59A97> 蝔见<E89D94><E8A781>𣂼<EFBFBD>餈鞱<E9A488>嚗峕<E59A97><E5B395>隞嗆芋<E59786><EFBFBD>霂閖<E99C82><EFBFBD>
### 2. 客户端程序 (`NaviGlassClient`)
**语言**: C++
* **多线程架构**: `CameraThread`, `AudioCaptureThread`, `IMUThread` 独立运行。
* **断网重连**:
* WebSocket 连接具备指数退避 (Exponential Backoff) 重连机制。
* *注: 底层 WiFi 接口需依赖 OS 脚本保活。*
* **音频流**: 20ms 分包 (320 samples @ 16kHz),确保低延迟传输。
---
### 8. <20><EFBFBD> I/O <20>躰秤憭<E7A7A4><E686AD> (Day 10) <20>?
#### <20><EFBFBD>
蝔见<EFBFBD>餈鞱<EFBFBD>蝥?10 蝘鍦<E89D98>撏拇<E6928F>嚗?- ALSA 餈𥪜<E9A488> I/O error
- `snd_pcm_recover()` <20><EFBFBD>憭梯揖
- 蝐餃<E89D90>頧祆揢<E7A586>躰秤嚗Ǒsize_t` vs `snd_pcm_sframes_t`嚗?- 蝻箏<E89DBB><E7AE8F>躰秤璉<E7A7A4><E79289>亙紡<E4BA99>?Segmentation fault
#### 閫<><E996AB><EFBFBD><EFBFBD>
**audio_capture.cpp - 霈曉<E99C88><E69B89>齿鰵<E9BDBF><EFBFBD><E598A5>?*嚗?```cpp
int err = snd_pcm_recover(m_pcm_handle, frames_read, 0);
if (err < 0) {
LOG_ERROR("[AudioCapture] Cannot recover: %s, attempting to reinitialize...",
snd_strerror(err));
// <20>喲𡡒霈曉<E99C88>
snd_pcm_close(m_pcm_handle);
m_pcm_handle = nullptr;
// 蝑匧<E89D91>撟園<E6929F><E59C92><EFBFBD>憪见<E686AA>
usleep(500000); // 500ms
if (!init()) {
LOG_ERROR("[AudioCapture] Failed to reinitialize device");
return -1; // 敶餃<E695B6>憭梯揖
}
LOG_INFO("[AudioCapture] Device reinitialized successfully");
return 0; // <20>祆活霂餃<E99C82>憭梯揖嚗䔶<E59A97>霈曉<E99C88>撌脫<E6928C>憭?}
```
**main.cpp - 蝐餃<E89D90>摰匧<E691B0><E58CA7><EFBFBD>霂舀<E99C82><E88880>?*嚗?```cpp
// 靽桀<E99DBD>蝐餃<E89D90><E9A483>躰秤
snd_pcm_sframes_t frames_read = mic.read(buffer, 160); // 雿輻鍂<E8BCBB>厩泵<E58EA9>瑞掩<E7919E>?
// 瘛餃<E7989B><E9A483>躰秤璉<E7A7A4><E79289>?if (frames_read < 0) {
LOG_ERROR("[AUD-CAP] Fatal error, exiting thread");
break; // 摰匧<E691B0><E58CA7><EFBFBD><EFBFBD>箇瑪蝔页<E89D94>銝滚蔣<E6BB9A><EFBFBD>隞𡝗芋<F0A19D97>?}
if (frames_read > 0) {
ws_aud.send_binary((uint8_t*)buffer, frames_read * 2);
}
```
#### 撉諹<E69289>蝏𤘪<E89D8F>
- <20>?蝔见<E89D94>蝔喳<E89D94>餈鞱<E9A488> 36+ 蝘𡜐<E89D98>靽桀<E99DBD><E6A180><EFBFBD>10 蝘鍦援皞<E68FB4><E79A9E>
- <20>?<3F>芸𢆡<E88AB8><EFBFBD><EFBCB7><EFBFBD><E7AE8F><EFBFBD><EFBFBD><EFBFBD><E59A97>霂蓥葉<E893A5>𣂼<EFBFBD><F0A382BC><EFBFBD> 3 甈?I/O <20>躰秤嚗?- <20>?<3F><EFBFBD>蝥輻<E89DA5>憭梯揖銝滚蔣<E6BB9A>?IMU <20><EFBFBD><E5B395>誩仍蝥輻<E89DA5>
---
### 9. WebSocket 撏拇<E6928F>靽桀<E99DBD> (Day 11) <20>?
#### <20><EFBFBD>
蝔见<EFBFBD><EFBFBD>?WebSocket 餈墧𦻖憭梯揖嚗?03 Forbidden嚗匧<E59A97>撏拇<E6928F>嚗?- <20>躰秤靽⊥<E99DBD>嚗䫤terminate called without an active exception`
- <20><EFBFBD> 1嚗䫤perform_handshake()` 憭梯揖<E6A2AF>?`m_recv_thread` <20>芸鍳<E88AB8>?- <20><EFBFBD> 2嚗䫤disconnect()` 隞滚<E99A9E>霂?`join()` <20>芸鍳<E88AB8><EFBFBD>蝥輻<E89DA5>
- <20><EFBFBD> 3嚗𡁏𦻖<F0A1818F>嗥瑪蝔钅獈憛𧼮銁 `recv()` <20>?`disconnect()` <20>湔𦻖<E6B994>喲𡡒 socket
#### 閫<><E996AB><EFBFBD><EFBFBD>
**ws_client.cpp - 甇<><EFBFBD><EFBC86>鱏撘<E9B18F>餈墧𦻖瘚<F0A6BB96><E7989A>**嚗?```cpp
void WSClient::disconnect() {
// 1. 霈曄蔭<E69B84><E894AD><EFBFBD>雿? m_running = false;
m_connected = false;
// 2. <20><EFBFBD><E785BE><EFBFBD><EFBFBD>剖葷
if (m_sockfd >= 0) {
uint8_t close_frame[] = {0x88, 0x00};
send(m_sockfd, close_frame, sizeof(close_frame), 0);
// 潃?<3F>喲睸靽桀<E99DBD>嚗𡁜<E59A97> shutdown 銝剜鱏<E5899C><EFBFBD><E9A483>?recv()
shutdown(m_sockfd, SHUT_RDWR);
}
// 3. <20>?join 摰鮋<E691B0>餈鞱<E9A488><E99EB1><EFBFBD>瑪蝔? if (m_recv_thread.joinable()) {
m_recv_thread.join();
}
// 4. <20>喲𡡒 socket
if (m_sockfd >= 0) {
close(m_sockfd);
m_sockfd = -1;
}
}
```
**<2A>喲睸蝏誯<E89D8F>**嚗?- `shutdown()` <20>臭誑銝剜鱏<E5899C><EFBFBD><E9A483>?`recv()` 靚<>
- 敹<><EFBFBD>?`join()` <20><EFBFBD><E6BBA9>?`shutdown()`
- 雿輻鍂 `joinable()` 璉<><E79289>亦瑪蝔𧢲糓<F0A7A2B2>血虾 join
---
### 10. <20><EFBFBD><E596B2><EFBFBD><EFBFBD><EFBFBD>帋縑摰䂿緵 (Day 11) <20>?
#### <20><>瘙?- <20><EFBFBD>蝟餌<E89D9F><E9A48C>芣𣈲<E88AA3><F0A388B2>𨺗憸煾<E686B8><E785BE><EFBFBD><EFBFBD>銝𠹺<E98A9D>嚗?- <20><><EFBFBD><E996AC><EFBFBD>圈𨺗憸烐偘<E78390><EFBFBD>TTS 霂剝𨺗<E5899D><F0A8BA97><EFBFBD>嚗?- <20>滚𦛚<E6BB9A><EFBFBD><EFBFBD> WebSocket 餈𥪜<E9A488>鈭諹<E988AD><E8ABB9>?PCM <20><EFBFBD><E596B2>唳旿
#### 閫<><E996AB><EFBFBD><EFBFBD>
**甇仿炊 1: <20><EFBFBD> WSClient <20><EFBFBD>鈭諹<E988AD><E8ABB9>嗆㺭<E59786><EFBFBD><E6A185>?*
```cpp
// ws_client.h - 瘛餃<E7989B>鈭諹<E988AD><E8ABB9>嗆㺭<E59786><EFBFBD><E6A185>?class WSClient {
public:
void poll_binary_messages(std::function<void(const uint8_t*, size_t)> callback);
private:
std::queue<std::vector<uint8_t>> m_binary_queue;
std::mutex m_binary_mutex;
};
// ws_client.cpp - recv_loop 靽嘥<E99DBD>鈭諹<E988AD><E8ABB9>嗆㺭<E59786>?if (opcode == OP_BINARY) {
std::lock_guard<std::mutex> lock(m_binary_mutex);
m_binary_queue.push(payload);
LOG_DEBUG("[WS] Received binary: %zu bytes", payload.size());
}
```
**甇仿炊 2: 銝餃儐<E9A483><EFBFBD><E888AB>鞾𨺗憸烐偘<E78390>?*
```cpp
// main.cpp - <20><EFBFBD><E598A5>𡝗碸憯啣膥嚗áw:1,0 - I2S <20>亙藁嚗?AudioPlayer speaker("hw:1,0", 16000, 1); // 16kHz, mono
bool speaker_enabled = speaker.init();
// 銝餃儐<E9A483>臭葉<E887AD>交𤣰撟嗆偘<E59786>暸𨺗憸?if (speaker_enabled) {
ws_aud.poll_binary_messages([&](const uint8_t* data, size_t size) {
if (size % 2 == 0) { // S16_LE <20>唳旿
size_t frames = size / 2;
speaker.write((const int16_t*)data, frames);
}
});
}
```
**<2A>喲睸蝏誯<E89D8F>**嚗?- WebSocket <20>?`recv_loop` 撌脩<E6928C><E884A9>函𡠺蝡讠瑪蝔衤葉餈鞱<E9A488>
- 鈭諹<E988AD><E8ABB9>嗆㺭<E59786><EFBFBD><EFBFBD><E69C9E><EFBFBD>隡𣳇<E99AA1><EFBFBD>銝餌瑪蝔?- 雿輻鍂 `hw:1,0`嚗㇆2S嚗㕑<E59A97><EFBFBD> `hw:0,0`嚗㇁udio Codec嚗?- <20><EFBFBD><E596B2><EFBFBD><EFBFBD><EFBFBD><EFBFBD>嚗?6kHz, mono, S16_LE
#### 撉諹<E69289>蝏𤘪<E89D8F>
- <20>?<3F>砍ㄟ<E7A08D><EFBFBD>憪见<E686AA><E8A781>𣂼<EFBFBD>嚗áw:1,0嚗?- <20>?<3F><EFBFBD><E596B2><EFBFBD><EFBFBD><EFBFBD>虜嚗<E8999C><EFBFBD><EFBFBD>?2000+ <20>唳旿<E594B3><E697BF><EFBFBD>
- <20>?TTS <20><EFBFBD><E596B2>剜𦆮撌脣<E6928C><E884A3><EFBFBD>Day 13 靽桀<E99DBD>嚗?
---
### 11. TTS 鈭衤辣敺芰㴓<E88AB0><EFBFBD>靽桀<E99DBD> (Day 13) <20>?
#### <20><EFBFBD>
TTS <20><EFBFBD><E596B2>剜𦆮<E5899C>剜鱏蝏剔賒嚗峕<E59A97><E5B395>?5-15 蝘埝<E89D98><E59F9D>剜𦆮<E5899C>牐葵摮梹<E691AE>
- 摰<E691B0>蝡舀𠯫敹梹<E695B9>`[AudioPlayer] Underrun occurred, recovering...`
- <20>滚𦛚<E6BB9A>冽𠯫敹梹<E695B9>TTS <20>烾𡢿<E783BE>?5-15 蝘?
#### <20><EFBFBD><E5ADB5><EFBFBD><EFBFBD>
`omni_client.py` 銝凋蝙<E5878B><EFBFBD>甇亥翮隞<E7BFAE>膥憭<E886A5><E686AD> Omni API <20><EFBFBD>嚗屸獈憛硺<E6869B><E7A1BA>港葵 asyncio 鈭衤辣敺芰㴓嚗?```python
# <20><EFBFBD><EFBFBD><E99A9E>
async def stream_chat(...):
completion = client.chat.completions.create(stream=True, ...)
for chunk in completion: # <20>?<3F>峕郊餈凋誨嚗屸獈憛硺<E6869B>隞嗅儐<E59785><EFBFBD>
yield OmniStreamPiece(...)
```
#### 閫<><E996AB><EFBFBD><EFBFBD>
**甇仿炊 1: Omni 摰<E691B0>蝡臬<E89DA1>甇亙<E79487>**
雿輻鍂 `threading.Thread` + `asyncio.Queue` 閫<><E996AB><EFBFBD>甇?API 靚<>鍂嚗?
```python
async def stream_chat(...):
queue = asyncio.Queue()
loop = asyncio.get_running_loop()
def _sync_stream():
"""<22>函𡠺蝡讠瑪蝔衤葉餈鞱<E9A488><E99EB1>峕郊 API 靚<>鍂"""
completion = client.chat.completions.create(stream=True, ...)
for chunk in completion:
piece = OmniStreamPiece(...)
loop.call_soon_threadsafe(queue.put_nowait, piece)
loop.call_soon_threadsafe(queue.put_nowait, None) # 蝏𤘪<E89D8F><F0A498AA><EFBFBD>
thread = threading.Thread(target=_sync_stream, daemon=True)
thread.start()
while True:
item = await queue.get() # <20>鮋獈憛䂿<E6869B>敺? if item is None:
break
yield item
```
**甇仿炊 2: 摰<E691B0>蝡?TTS 憸<><E686B8><EFBFBD>脫㦤<E884AB>?*
```cpp
// main.cpp - TTS 憸<><E686B8><EFBFBD><EFBFBD><E884A3>?static std::vector<int16_t> tts_buffer;
static const size_t PRE_BUFFER_FRAMES = 16000; // 1蝘㘾<E89D98>蝻枏<E89DBB>
static const size_t MIN_PLAY_FRAMES = 8000; // 0.5蝘埝<EFBFBD>撠𤩺偘<EFBFBD><EFBFBD><EFBFBD>?static bool is_buffering = true;
// <20>交𤣰<E4BAA4>嗉蕭<E59789><EFBFBD>蝻枏<E89DBB><E69E8F>?ws_aud.poll_binary_messages([&](const uint8_t* data, size_t size) {
const int16_t* samples = (const int16_t*)data;
size_t frames = size / 2;
tts_buffer.insert(tts_buffer.end(), samples, samples + frames);
// 蝘舐敞頞喳<E9A09E><E596B3><EFBFBD>憪𧢲偘<F0A7A2B2>? if (is_buffering && tts_buffer.size() >= PRE_BUFFER_FRAMES) {
is_buffering = false;
}
});
// <20>剜𦆮<E5899C>嗡蝙<E597A1><EFBFBD>撠𤩺鸌甈? if (!is_buffering && tts_buffer.size() >= MIN_PLAY_FRAMES) {
size_t play_frames = std::min(tts_buffer.size(), (size_t)1600); // 100ms
speaker.write(tts_buffer.data(), play_frames);
tts_buffer.erase(tts_buffer.begin(), tts_buffer.begin() + play_frames);
}
```
**甇仿炊 3: <20>滚𦛚<E6BB9A><EFBFBD><E588B8><EFBFBD>箔耨憭?*
```python
# app_main.py - lifespan <20>閗繮 CancelledError
@asynccontextmanager
async def lifespan(app):
# ... <20>臬𢆡<E887AC><EFBFBD> ...
try:
yield
except asyncio.CancelledError:
pass # Ctrl+C 甇<>虜銵䔶蛹
finally:
print("[LIFESPAN] 摨𠉛鍂<F0A0899B>喲𡡒摰峕<E691B0>")
# 撘箏<E69298><E7AE8F><EFBFBD><EFBFBD>箇瑪蝔? def _force_exit():
time.sleep(0.5)
os._exit(0)
threading.Thread(target=_force_exit, daemon=True).start()
```
#### 撉諹<E69289>蝏𤘪<E89D8F>
| <20><><EFBFBD> | 靽桀<E99DBD><E6A180>?| 靽桀<E99DBD><E6A180>?|
|------|--------|--------|
| TTS <20>烾𡢿<E783BE>?| 5-15 蝘?| ~2 蝘?|
| 擐𡝗活<F0A19D97>剜𦆮撱嗉<E692B1> | 蝡见朖 | ~1 蝘𡜐<E89D98><EFBFBD><E686B8><EFBFBD><EFBFBD> |
| Underrun 憸𤑳<E686B8> | 瘥誩<E798A5><EFBFBD>甈?| <20><EFBFBD> |
| <20>滚𦛚<E6BB9A><EFBFBD><E588B8><EFBFBD>?| <20><>絲 | 甇<>虜餈𥪜<E9A488> |
#### <20><EFBFBD><E39B96><EFBFBD>
- TTS 隞齿<E99A9E>頧餃凝<E9A483>剔賒嚗㇉mni API <20><EFBFBD><E6BB9A><EFBFBD>
- 撱箄悅<E7AE84><E68285><EFBFBD>?HTTP `/stream.wav` 璅<E79285><EFBFBD><E59A97><EFBFBD>?ESP32S3 摰䂿緵嚗?
---
## <20><> <20>喲睸<E596B2>𣂼停
### <20><><EFBFBD><EFBFBD><E88890>?1. <20>?隞𡡞妟摰䂿緵 GPIO 璅⊥<E79285> SPI 撽勗𢆡
2. <20>?閫<><E996AB> I2S Device Tree <20>滨蔭<E6BBA8><EFBFBD>
3. <20>?ICM-42688 SPI 撽勗𢆡摰峕㟲摰䂿緵
4. <20>?MPP 獢<>沲摰峕㟲<E5B395><E39FB2><EFBFBD>嚗ĀI<C480>𥕜SP<53>栐ENC嚗?5. <20>?GC2083 100% <20>閗繮<E99697>𣂼<EFBFBD><F0A382BC>?6. <20>?<3F><EFBFBD><E8B9B1><EFBFBD>?60+ 摨㮖<E691A8>韏𡝗<E99F8F><F0A19D97>?7. <20>?撌亙<E6928C><E4BA99><EFBFBD>蝵桃<E89DB5><EFBFBD><E98A9D><EFBFBD>
8. <20>?3.9MB 銝餌<E98A9D>摨讐<E691A8>霂烐<E99C82><E78390>?9. <20>?<3F><EFBFBD><E596B2>躰秤<E8BAB0>芸𢆡<E88AB8><EFBFBD><EFBCB7><EFBFBD>
10. <20>?**WebSocket 撏拇<E6928F><E68B87><EFBFBD>靽桀<E99DBD>**
11. <20>?**<2A><EFBFBD><E596B2><EFBFBD><EFBFBD><EFBFBD>帋縑摰䂿緵**
12. <20>?**TTS 鈭衤辣敺芰㴓<E88AB0><EFBFBD>靽桀<E99DBD>**嚗㇉mni 摰<E691B0>蝡臬<E89DA1>甇亙<E79487>嚗?13. <20>?**摰<E691B0>蝡?TTS 憸<><E686B8><EFBFBD>脫㦤<E884AB>?*
14. <20>?**憿寧𤌍隞<F0A48C8D><E99A9E><EFBFBD>西澈** (蝘駁膄 19 銝芸<E98A9D>雿蹱<E99BBF>隞?
15. <20>?**<2A>鮋獈憛噼歲撣扳㦤<E689B3>?* (撖潸⏛ FPS 0.5 -> 10.0)
### <20><EFBFBD><E689AF><EFBFBD><EFBFBD>
- <20><EFBFBD><E596B2><EFBFBD><EFBFBD><E79485><EFBFBD>16kHz <20>?- <20><><EFBFBD>憭游葷<E6B8B8><E891B7><EFBFBD>20fps <20>?- JPEG <20>讠憬韐券<E99F90>嚗?0 <20>?- SPI <20>帋縑<E5B88B>笔漲嚗鰺500kHz <20>?- IMU <20>唳旿<E594B3><E697BF><EFBFBD>1kHz <20>?- 蝻𤥁<E89DBB>鈭抒<E988AD>憭批<E686AD>嚗?.9MB <20>?- **<2A><EFBFBD>餈鞱<E9A488>撉諹<E69289>嚗𡁏<E59A97><F0A1818F>㗇芋<E39787>?00%<25><EFBFBD>** <20>?- **TTS <20>烾𡢿<E783BE><EFBFBD>~2 蝘?*嚗<>耨憭滚<E686AD> 5-15 蝘𡜐<E89D98><F0A19C90>?
---
## <20><> 蝏誯<E89D8F>蝘舐敞
### Device Tree <20>滨蔭
- `pinctrl` <20><EFBFBD>嚗𡁏<E59A97>銝芸<E98A9D><E88AB8>𡁶𡠺蝡贝<E89DA1><E8B49D>?- `function` vs `allwinner,function`嚗帋<E59A97><E5B88B><EFBFBD><EFBFBD><EFBFBD><E586BD><EFBFBD><EFBFBD><EFBFBD>?- 撘閗<E69298>憭滨鍂<E6BBA8><EFBFBD>嚗𡁻<E59A97><EFBFBD> `/sys/kernel/debug/pinctrl` <20>埝䰻
### GPIO <20><EFBFBD>
- `/sys/class/gpio` <20>亙藁雿輻鍂
- 撖澆枂 <20>?霈曄蔭<E69B84><EFBFBD> <20>?霂餃<E99C82><E9A483>?- GPIO 蝻硋噡霈∠<E99C88>嚗䫤(bank - 'A') * 32 + offset`
### SPI <20>讛悅
- Mode 0嚗鋴POL=0, CPHA=0
- <20><EFBFBD>嚗鐝OSI<53><49><EFBFBD> <20>?SCLK銝𠰴<E98A9D>瘝?<3F>?MISO<53><4F>
- ICM-42688 霂餃<E99C82>嚗𡁻<E59A97>摮𡑒<E691AE> 0x80|撖<><E69296><EFBFBD>典𧑐<E585B8><F0A79190>
### ALSA <20><EFBFBD>
- 餈鞱<E9A488><E99EB1><EFBFBD>蝵?vs 蝻𤥁<E89DBB><F0A4A581><EFBFBD>蝵?- `amixer`<><E99D9A><EFBFBD><EFBFBD>撌?- MIC Gain 靚<><E99D9A><EFBFBD><EFBFBD>
### MPP 獢<>
- <20><EFBFBD><E598A5>㚚◇摨𧶏<E691A8>VI <20>?ISP <20>?VENC
- 蝻枏<E89DBB><E69E8F><EFBFBD><E7AE94>𣇉<EFBFBD><F0A38789>?- JPEG 蝻𣇉<E89DBB>韐券<E99F90>銝擧<E98A9D>隞嗅之撠𤩺<E692A0>銵?
### 鈭文<E988AD>蝻𤥁<E89DBB>
- <20><EFBFBD><E8B9B1><EFBFBD><EFBFBD>暹𦻖憿箏<E686BF><EFBFBD><E996AB>
- Cedar 摨㮖<E691A8>韏硋<E99F8F>蝟?- 撌亙<E6928C><E4BA99>曇楝敺<E6A59D><E695BA><EFBFBD>䲮瘜?
---
## <20>𩤃<EFBFBD> <20><EFBFBD><E39B96><EFBFBD>銝𦒘<E98A9D><F0A69298>𡝗䲮<F0A19D97>?
### <20><><EFBFBD>舀䔝蝝<EFBCB8>?1. **蝖砌辣 SPI**
- 敶枏<E695B6>嚗鎭PIO 璅⊥<E79285> (~500kHz)
- <20><EFBFBD><E887AD><EFBFBD>雿輻鍂蝖砌辣 SPI (<28>衽Hz)
- <20><EFBFBD>嚗𡁻<E59A97>雿?CPU <20>删鍂
2. **I2C <20><EFBFBD>**
- 敶枏<E695B6>嚗𡁜歇<F0A1819C><EFBFBD>
- <20>舀䔝蝝<E89D9D><EFBD87><EFBFBD><E5AF9E><EFBFBD><E598A5><EFBFBD><E7A18B>? - <20><EFBFBD>嚗𡁏凒<F0A1818F><E58792><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>?
3. **musl 撌亙<E6928C><E4BA99>?*
- 敶枏<E695B6>嚗帋蝙<E5B88B>?glibc
- <20><EFBFBD>蝘鳴<E89D98><EFBFBD><E996AB> musl 撌亙<E6928C><E4BA99>? - <20><EFBFBD>嚗𡁜<E59A97>撠誩虾<E8AAA9><EFBFBD><E689AF><EFBFBD>辣雿梶妖
### <20><EFBFBD>隡睃<E99AA1>
- GPIO 璅⊥<E79285> SPI <20>笔漲隡睃<E99AA1><EFBFBD>𤌍<EFBFBD>?1-2MHz嚗?- <20><><EFBFBD>雿輻鍂隡睃<E99AA1>
- 憭𡁶瑪蝔贝<E89D94>頧賢<E9A0A7>銵?
### 隞<><E99A9E>韐券<E99F90>
- 蝏煺<E89D8F>瘜券<E7989C>憌擧聢
- 瘛餃<E7989B><E9A483><EFBFBD>瘚贝<E7989A>
- <20>躰秤憭<E7A7A4><E686AD><EFBFBD><E691B0>
---
---
### 12. 憿寧𤌍皜<F0A48C8D><E79A9C>銝?TTS 瘚<><E7989A> (Day 14) <20>?
#### 隡睃<E99AA1><E79D83><EFBFBD>
1. **HTTP <20><><EFBFBD><EFBFBD><EFBFBD>靽桀<E99DBD>**嚗𡁜<E59A97><F0A1819C>?HTTP 20ms pacing 敺芰㴓<E88AB0><EFBFBD>鈭?WebSocket <20><EFBFBD><E785BE><EFBFBD>撖潸稲 TTS <20>⊿▼<E28ABF><E296BC>耨憭齿䲮獢<E4B2AE><E78DA2>撠?HTTP 撟踵偘蝘餉秐<E9A489>𤾸蝱隞餃𦛚 `asyncio.create_task()`<60>?2. **憿寧𤌍<E5AFA7>西澈**嚗𡁶宏<F0A181B6>?HTTP TTS <20><EFBFBD><EFBFBD><E99A9E><EFBFBD>𦠜<EFBFBD>霂閗<E99C82><E99697>穿<EFBFBD><E7A9BF><EFBFBD><EFBFBD><EFBFBD> 19 銝芥<E98A9D>?3. **<EFBFBD><EFBFBD><EFBFBD>**嚗関TS <20>剜𦆮摰<F0A686AE><E691B0><EFBFBD><E7989A>嚗峕<E59A97><E5B395>剔賒<E58994>?
### 13. 撖潸⏛璅<E79285><E288AA>⊿▼靽桀<E99DBD> (Day 15) <20>?
#### <20><EFBFBD><E6A185>啗情
撖潸⏛璅<EFBFBD><EFBFBD><EFBFBD><EFBFBD>嚗𤅎PS 隞?10.0 <20><EFBFBD><E6B8B2>?0.5-1.5嚗𣬚𤫇<EFBFBD>㨃憿踴<EFBFBD>?
#### <20><><EFBFBD>舀䲮獢<E4B2AE><E78DA2><EFBFBD>鮋獈憛噼歲撣扳㦤<E689B3>?**<2A>煺誨<E785BA><E8AAA8>䔮憸?*嚗䫤await loop.run_in_executor` <20><EFBFBD><E8B39C>函瑪蝔𧢲<E89D94><F0A7A2B2><EFBFBD>嚗䔶<E59A97>銝餃儐<E9A483><EFBFBD>蝑匧<E89D91>蝏𤘪<E89D8F>餈𥪜<E9A488><EFBFBD><EFBFBD>港葡銵屸獈憛𠺶<E6869B>?
**靽桀<E99DBD><E6A180><EFBFBD>**嚗?1. **Fire-and-Forget 璅<E79285>**嚗帋蜓敺芰㴓<E88AB0>𣂷漱隞餃𦛚<E9A483>𡒊<EFBFBD><F0A1928A>喟誧蝏哨<E89D8F>銝滨<E98A9D>敺?`await`<60>?2. **<EFBFBD><EFBFBD><EFBFBD>啣葷隡睃<EFBFBD>**嚗𡁜<E59A97><F0A1819C>𨀣<EFBFBD><F0A880A3>啣葷銝𥪯<E98A9D><EFBFBD>撣扳𧊋憭<F0A78A8B><E686AD>摰䕘<E691B0><E49598><EFBFBD><E8BAB0>?Pending 撣改<E692A3>蝖桐<E89D96><EFBFBD><E686AD><EFBFBD><EFBFBD><EFBFBD>唳㺭<E594B3><EFBFBD>?3. **蝏𤘪<E89D8F>憭滨鍂**嚗𡁜銁<F0A1819C>𤾸蝱憭<E89DB1><E686AD>摰峕<E691B0><E5B395><EFBFBD>撟踵偘銝𠹺<E98A9D>撣抒<E692A3><EFBFBD>瘚讠<E7989A><E8AEA0><EFBFBD>靽脲<E99DBD> UI <20>瑟鰵<E7919F><E9B0B5>蛹 10FPS<50>?
**<EFBFBD><EFBFBD><EFBFBD>**嚗?- 摰<E691B0>蝡舫<E89DA1><E888AB><EFBFBD><EFBFBD>10 FPS (1280x720)
---
### 14. <20>箔辣<E7AE94><EFBFBD>銝舘䌊<E88898>臬𢆡<E887AC>滨蔭 (Day 16) <20>?
#### <20><EFBFBD> 1嚗𡁜𢆡<F0A1819C><F0A286A1><EFBFBD>亙膥頝臬<E9A09D>
蝔见<EFBFBD>蝻𤥁<EFBFBD><EFBFBD>嗡蝙<EFBFBD>?`/lib32/ld.so.1`嚗䔶<EFBFBD><EFBFBD><EFBFBD>烐踎摰鮋<EFBFBD>頝臬<EFBFBD>銝?`/lib/ld-musl-riscv32.so.1`<EFBFBD><EFBFBD>?`not found` <20>躰秤<E8BAB0>?
**閫<><E996AB><EFBFBD><EFBFBD>**嚗?```makefile
# src/Makefile 瘛餃<E7989B>
LDFLAGS += -Wl,--dynamic-linker=/lib/ld-musl-riscv32.so.1
```
#### <20><EFBFBD> 2嚗朞䌊<E69C9E>臬𢆡<E887AC><EFBFBD><E5AF9E>㗇𥋘
| <20><EFBFBD> | 蝏𤘪<E89D8F> | <20><EFBFBD> |
|------|------|------|
| `/etc/init.d/rc.final` | <20>?憭梯揖 | 撖潸稲 SD <20><EFBFBD>瘚见仃韐伐<E99F90><E4BC90><EFBFBD><EFBFBD><EFBFBD><E6BB9A>箔辣 |
| `crontab @reboot` | <20>?憭梯揖 | BusyBox crond 銝齿𣈲<E9BDBF><F0A388B2>迨霂剜<E99C82> |
| `/etc/init.d/avaota` + `load_script.conf` | <20>?<3F>𣂼<EFBFBD> | 蝔喳<E89D94><E596B3><EFBFBD> |
#### <20>𣂼<EFBFBD><F0A382BC><EFBFBD>霂西圾
**1. <20>𥕦遣 init <20>𡁏𧋦**嚗?```bash
cat > /etc/init.d/avaota << 'EOF'
#!/bin/sh /etc/rc.common
START=99
start() {
sleep 15
ulimit -c 0
/mnt/UDISK/app/avaota_client > /tmp/avaota.log 2>&1 &
}
stop() {
killall avaota_client
}
EOF
chmod +x /etc/init.d/avaota
```
**2. 瘛餃<E7989B><E9A483>啣鍳<E595A3><EFBFBD>銵?*嚗?```bash
echo "avaota" >> /etc/init.d/load_script.conf
```
#### <20>喲睸蝏誯<E89D8F>
- **<2A>臬𢆡撱嗉<E692B1>**嚗𡁻<E59A97>閬?15 蝘垍<E89D98><EFBFBD>𨺗憸𤑳頂蝏笔<E89D8F>憪见<E686AA>嚗? 蝘雴<E89D98><EFBFBD><E686AD>
- **蝳<>鍂 core dump**嚗䫤ulimit -c 0` <20>脫迫蝔见<E89D94>撏拇<E6928F><E68B87>烐說 overlay <20><>
- **WiFi <20>芸𢆡餈墧𦻖**嚗𡁶頂蝏煺<E89D8F>靽嘥<E99DBD><E598A5>滨蔭<E6BBA8>?`/etc/wifi/wifimg.config`嚗峕<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>砌葉<EFBFBD>滨蔭
- **<2A><EFBFBD> rc.final**嚗帋<E59A97>撟脫贋 SD <20>?mdev <20><EFBFBD><E598A5>𡝗<EFBFBD>蝔?
#### <20><EFBFBD><E7A589>滚𦛚<E6BB9A><EFBFBD>蝵?
靽格㺿 `main.cpp` <20><EFBFBD> frp <20><><EFBFBD>蝛輸<E89D9B>𧶏<EFBFBD>
```cpp
// 靽格㺿<E6A0BC>?const char* SERVER_HOST = "192.168.110.188";
// 靽格㺿<E6A0BC>?const char* SERVER_HOST = "8.148.25.142";
```
**frpc.toml <20>滨蔭**嚗?```toml
serverAddr = "8.148.25.142"
serverPort = 7000
auth.token = "your_token"
[[proxies]]
name = "avaota_server"
type = "tcp"
localIP = "127.0.0.1"
localPort = 8081
remotePort = 8081
[[proxies]]
name = "avaota_imu_udp"
type = "udp"
localIP = "127.0.0.1"
localPort = 12345
remotePort = 12345
```
> **Day 17 銵亙<E98AB5>**嚗𡁜<E59A97>憿餅溶<E9A485>?UDP 12345 隞<><E99A9E><EFBFBD><EFBFBD>?IMU <20>唳旿<E594B3><EFBFBD><E4ADBE><EFBFBD><E69C9E><EFBFBD>隡㰘<E99AA1><E3B098><EFBFBD><EFBFBD><EFBFBD><EFBFBD><E59C92><EFBFBD>蝵烐<E89DB5><E78390><E288AA><EFBFBD>憓坔<E68693><E59D94>?UDP 12345 蝡臬藁<E887AC>?- <20>滚𦛚<E6BB9A><EFBFBD><E585B8><EFBFBD><EFBFBD>~15 FPS (<28><EFBFBD><E39787>拍鍂)
- 撖潸⏛雿㯄<E99BBF>嚗𡁏<E59A97><F0A1818F><EFBFBD><EFBFBD><EFBFBD>⊿▼
---
## <20><> <20><><EFBFBD><EFBFBD><EFBFBD>皞?
### 蝖砌辣<E7A08C><E8BEA3>
- [<5B><EFBFBD> V821 Datasheet](https://www.aw-ol.com)
- [ICM-42688 Datasheet](https://invensense.tdk.com)
- [MAX98357A Datasheet](https://www.maximintegrated.com)
- [GC2083 Datasheet](http://www.galaxycore.com.cn)
### 頧臭辣<E887AD><E8BEA3>
- [ALSA Documentation](https://www.alsa-project.org)
- [MPP Framework Guide](https://github.com/allwinner)
- [Linux GPIO Subsystem](https://www.kernel.org/doc/html/latest/driver-api/gpio/)
### 撘<><E69298>烐𠯫敹?- [x] [Day1.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day1.md) - SDK蝻𤥁<E89DBB><F0A4A581><EFBFBD><E887AC>剖遣
- [x] [Day2.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day2.md) - 蝵𤑳<E89DB5>摨梶<E691A8>霂?- [x] [Day3.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day3.md) - I2S<32><EFBFBD>颲枏枂
- [x] [Day4.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day4.md) - 暻血<E69ABB>憌?IMU
- [x] [Day5.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day5.md) - GC2083<38><33><EFBFBD>憭?- [x] [Day6.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day6.md) - 蝖砌辣撉諹<E69289>
- [x] [Day7.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day7.md) - 撌亙<E6928C><E4BA99><EFBFBD>蝵?- [x] [Day8.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day8.md) - <20><EFBFBD>蝻𤥁<E89DBB><F0A4A581>𣂼<EFBFBD>
- [x] [Day9.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day9.md) - **musl 撌亙<E6928C><E4BA99>曆耨憭?+ <20><EFBFBD>瘚贝<E7989A><E8B49D><EFBFBD>** <20>?- [x] [Day10.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day10.md) - **<2A><EFBFBD> I/O <20>躰秤靽桀<E99DBD> + 蝔见<E89D94>蝔喳<E89D94><E596B3><EFBFBD><E689B3>?* <20>?- [x] [Day11.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day11.md) - **WebSocket 撏拇<E6928F>靽桀<E99DBD> + <20><EFBFBD><E596B2>剜𦆮摰䂿緵** <20>?- [x] [Day12.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day12.md) - **<2A><EFBFBD><E596B2>䂿㴓<E482BF><E3B493><EFBFBD> + <20>砍ㄟ<E7A08D>冽毽<E586BD>喳膥靽桀<E99DBD>** <20>?- [x] [Day13.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day13.md) - **TTS 鈭衤辣敺芰㴓<E88AB0><EFBFBD>靽桀<E99DBD> + 摰<E691B0>蝡舫<E89DA1>蝻枏<E89DBB>** <20>?- [x] [Day14.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day14.md) - **憿寧𤌍皜<F0A48C8D><E79A9C> + TTS 瘚<><E7989A><EFBFBD>剜𦆮** <20>?- [x] [Day15.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day15.md) - **撖潸⏛璅<E79285><E288AA><EFBFBD>隡睃<E99AA1> (頝喳葷<E596B3><EFBFBD>)** <20>?- [x] [Day16.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day16.md) - **<2A>箔辣<E7AE94><EFBFBD>銝舘䌊<E88898>臬𢆡<E887AC>滨蔭** <20>?- [x] [Day17.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day17.md) - **摰文<E691B0>瘚贝<E7989A>+<2B><EFBFBD>霂剝𨺗靽桀<E99DBD>+IMU UDP隞<50><E99A9E>** <20>?- [ ] [Day19.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day19.md) - **隡㰘<E99AA1>隡睃<E99AA1> + Python GIL <20><EFBFBD>霂𦠜鱏** <20>𩤃<EFBFBD>
---
### 15. Python GIL <20><EFBFBD><E689AF><EFBFBD>霂𦠜鱏 (Day 19) <20>𩤃<EFBFBD>
#### <20><EFBFBD><E6A185>𤑳緵
<EFBFBD>滚𦛚<EFBFBD><EFBFBD>霂閧𤩅<EFBFBD>枏紡<EFBFBD>芣𧒄<EFBFBD>𤑳緵<EFBFBD><EFBFBD><EFBFBD><EFBFBD>嚗?- **GPU <20>拍鍂<E68B8D><E98D82><EFBFBD> 7%** - 餈靝<E9A488>鈭𡡞<E988AD><F0A1A19E>?- **Python 餈𤤿<E9A488><F0A4A4BF>删鍂 120% CPU** - 雿<><E99BBF><EFBFBD><E288AA>?CPU 隞?3%嚗?6 <20>詨⏚<E8A9A8><EFBFBD><E587BD><EFBFBD><EFBFBD>嚗?- **撣批<E692A3><E689B9>?FPS 隞?3-4 撣?* - <20>駁𢒰<E9A781>⊿▼銝仿<E98A9D>
#### <20><EFBFBD><E5ADB5><EFBFBD><EFBFBD>
```python
# app_main.py 蝚?237 銵?frame_processing_executor = ThreadPoolExecutor(max_workers=3, ...)
```
`ThreadPoolExecutor` <20>?Python GIL (<28><EFBFBD><EFBFBD><E996AB><EFBFBD><EFBFBD>) <20>𣂼<EFBFBD>嚗峕<E59A97>瘜閧<E7989C><EFBFBD>僎銵<E5838E><E98AB5> CPU 撖<><E69296><EFBFBD>衤遙<E8A1A4><EFBFBD>JPEG 蝻𤥁圾<F0A4A581><E59CBE><EFBFBD><EFBFBD><EFBFBD><EFBFBD>暹葡<E69AB9><EFBFBD><E6A992>?
#### 閫<><E996AB><EFBFBD><EFBFBD>
雿輻鍂 **PyNvJpeg** 撠?JPEG 蝻𤥁圾<F0A4A581><E59CBE><EFBFBD>?GPU嚗?
```bash
pip install pynvjpeg
```
```python
import pynvjpeg as nj
_nvjpeg = nj.NvJpeg()
def gpu_decode(jpeg_bytes):
return _nvjpeg.decode(jpeg_bytes)
def gpu_encode(image, quality=85):
return _nvjpeg.encode(image, quality)
```
**<EFBFBD><EFBFBD>?*嚗尠<E59A97> <20><EFBFBD><E88AB8><EFBFBD><EFBFBD><E695BA><EFBFBD><EFBFBD><E4BA99>?
---
**<EFBFBD><EFBFBD><EFBFBD>擧凒<EFBFBD>?*嚗?025-12-23 (Day 19 - 隡㰘<E99AA1>隡睃<E99AA1> + GIL <20><EFBFBD>霂𦠜鱏)
**憿寧𤌍<E5AFA7><EFBFBD>?*嚗尠<E59A97>儭?**99% 摰峕<E691B0><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><E695BA><EFBFBD>?(Python GIL <20><EFBFBD>)**
**Day 19 <20>湔鰵**嚗関urboJPEG 隡睃<E99AA1><E79D83><EFBFBD><EFBFBD><E89189><EFBFBD>雿㮖耨憭溻<E686AD><E6BABB>IL <20><EFBFBD>霂𦠜鱏
**敺<><E695BA><EFBFBD>?*嚗䥪yNvJpeg GPU JPEG <20>𣳇<EFBFBD><EFBFBD><E9BA84>TS <20><EFBFBD>靽桀<E99DBD>
## 🛠️ 部署与运维
* **内网穿透**: FRP (Fast Reverse Proxy)。将本地端口 8081 (WS) 和 12345 (UDP) 暴露到公网服务器。
* **进程管理**: PM2 (Server), Systemd/Init.d (Client)。
* **可视化监控**: Web Dashboard (`/static/index.html`) 提供实时第一视角画面、IMU 姿态、AI 对话日志和系统状态监控。

View File

@@ -2,21 +2,22 @@
**项目**NaviGlass项目开发
**平台**:全志 V821 (32-bit RISC-V)
**更新时间**2026-01-05
**整体进度**100%(室内导盲模型训练优化完成
**更新时间**2026-01-06
**整体进度**100%(室内导盲系统优化交付
## 📖 快速导航
| 章节 | 说明 |
|------|------|
| [已完成任务](#-已完成任务) | Day 1-27 完成的功能 |
| [待验证问题](#-待验证问题-day-20-优化后) | 当前需要验证的修复 |
| [已完成任务](#-已完成任务) | Day 1-28 完成的功能 |
| [后续规划](#-后续规划与遗留问题) | 待办 / 债务 / 未来展望 |
| [进度统计](#-进度统计) | 各模块完成度 |
| [时间线](#-时间线) | 开发时间线 |
| [里程碑](#-里程碑) | 关键节点回顾 |
| [时间线](#-时间线) | 开发流水账 |
**相关文档**
- [实现计划详情](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/implementation_plan_complete.md)
- [Day 日志](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/) (Day1-Day27)
- [Day 日志](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/) (Day1-Day28)
---
@@ -413,37 +414,39 @@
- [x] **跨平台路径修复**: 解决 Windows 生成 `data.yaml` 在 Linux 报错问题
- [x] **文档同步**: 更新《训练指南》至 V2 (22类 + 路径注意)
---
## ⏳ 待完成任务
### 性能评估
- [ ] CPU 使用率监控(目标: <80%
- [ ] 内存使用监控(目标: <100MB
- [x] 摄像头实际帧率: 客户端 10 FPS, 服务器处理 ~15 FPS
- [x] IMU 采样率: 10Hz
### 稳定性验证
- [ ] 长时间运行测试5分钟基础 / 1小时稳定性 / 内存泄漏检查)
- [ ] 异常恢复测试(断网重连 / 设备异常恢复)
- [ ] 压力测试(高负载 / 边界条件)
### 室内导航优化与 Bug 修复 (Day 28)
- [x] **模型精准化**
- [x] 切换至 `yolo11l-seg-indoor14.engine` (14类)
- [x] 更新 `workflow_indoor.py` 类别映射
- [x] 地面检测率提升,解决无路误报
- [x] **语音播报修复**
- [x] **强制首播**: 解决初始无路提示被节流的问题
- [x] **即时反馈**: 启动命令后立即播报“正在寻找可行走区域”
- [x] **音频 Pop 消除**: 增加静音填充至 200ms
- [x] **崩溃修复**: 移除 undefined class 引用
- [x] **终端 WiFi 诊断**
- [x] 确认断连不可恢复为 OS 层面问题
- [x] 建议部署 `wifi_check.sh` 守护脚本
---
## 🔮 未来优化方向
## 🛤️ 后续规划与遗留问题
### 性能优化
- [ ] 内存使用优化
- [ ] CPU 负载平衡
- [ ] 网络带宽管理
- [ ] 功耗优化
### 🔴 优先待办 (Priority)
- [ ] **稳定性验证**: 长时间运行测试5分钟基础 / 1小时稳定性 / 内存泄漏检查)
- [ ] **异常恢复**: 模拟断网重连、设备异常恢复等边界条件
- [ ] **WiFi 保活**: 部署终端脚本 `wifi_check.sh` 解决断网不重连问题
### 硬件升级探索
- [ ] 硬件 SPI 替代 GPIO 模拟提升IMU速度
- [ ] I2C 通信方案重新探索
### 🟠 技术债务 (Debt)
1. **音频底噪**: 需探索软件降噪 (AEC/ANS) 以消除回环底噪
2. **硬件屏蔽**: 扬声器电流声需硬件改进(磁环/布线)
3. **代码规范**: 补充单元测试,统一注释风格
### 🔵 长期探索 (Future)
- [ ] **硬件升级**: 尝试硬件 SPI 替代 GPIO 模拟(提升 IMU 采样率)
- [ ] **I2C 攻坚**: 重新探索 ICM-42688 的 I2C 通信模式
- [ ] **能效优化**: 内存占用优化与 CPU 负载平衡
---
## 📊 进度统计
@@ -474,7 +477,7 @@
| 导航模式 | 100% | ✅ Day 15 修复 |
| 服务器优化 | 100% | ✅ Day 18-19 完成 |
| AI 交互管道 | 100% | ✅ Day 21-22 修复 (GLM-4V/SenseVoice) |
| 室内导盲模型 | 100% | ✅ Day 25 训练完成 |
| 室内导盲模型 | 100% | ✅ Day 28 优化交付 (14类模型) |
---
@@ -595,41 +598,16 @@
- **稳定性修复**: 彻底解决 FP16 训练 NaN 问题 (AMP=False)
- **完整性验证**: 数据集健康检查 100% 通过
---
## 📝 技术债务
### 需要优化的地方
1. **GPIO 模拟性能**: 当前 SPI 速度 ~500kHz可优化到 1-2MHz
2. **错误处理**: 网络断开重连、设备异常恢复、日志完善
3. **代码规范**: 需要统一注释风格、需要单元测试
### 遗留问题
1. **音频底噪**: 开启 Playback Switch 可能引入回环底噪,需探索软件降噪 (AEC/ANS)
2. **扬声器杂音** (Day 17): 确认为硬件问题,需加磁环或重新布线
3. **I2C 问题**: ICM-42688 I2C 模式未解决,保留为技术探索方向
4. **硬件 SPI**: 未尝试使用硬件 SPI可作为性能优化方向
### Milestone 24: 室内导盲系统优化交付 ✅
**时间**: Day 28
**成果**:
- **高精度感知**: 切换 14 类模型,地面检测率/避障精准度显著提升
- **体验升级**: 修复无路提醒节流 Bug消除音频 Pop 噪声,交互响应即时化
- **系统诊断**: 明确终端 WiFi 断连根因并给出自动化运维方案
---
## 🛠️ 开发工具与资源
### 必备工具
- [x] WinSCP - 文件传输
- [x] PuTTY - SSH 终端
- [x] PhoenixSuit - 固件烧录
- [x] VS Code - 代码编辑
### 参考文档
- [x] [全志 V821 Datasheet](https://www.aw-ol.com)
- [x] [ICM-42688 Datasheet](https://invensense.tdk.com)
- [x] [MAX98357A Datasheet](https://www.maximintegrated.com)
- [x] [ALSA Documentation](https://www.alsa-project.org)
### 开发笔记
- [Day1.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day1.md) ~ [Day23.md](file:///d:/CodingProjects/Antigravity/NaviGlass/Docs/DevLogs/Day23.md)
---
## 📅 时间线
@@ -660,15 +638,4 @@ Day 24: YOLOE模型训练 ✅ 完成 (问题诊断+数据集筛选)
Day 25: 室内导盲模型完成 ✅ 完成 (训练+验证通过)
Day 26: 室内导盲服务器集成 ✅ 完成 (IndoorNavigator工作流)
Day 27: 模型训练优化与完善 ✅ 完成 (NaN修复+22类适配+路径修复)
```
---
**🎯 当前状态100% 完成!室内导盲系统已集成并可使用**
## ⚠️ 待验证问题
*(当前无遗留问题,所有已知 BUG 均已修复)*
Day 28: 室内导航优化与修复 ✅ 完成 (14类模型+语音修复+音频Pop修复)

View File

@@ -10,7 +10,7 @@
WebSocket: 用于音频/视频流的实时双向传输 (websockets 库)
HTTP: 用于控制指令和状态查询
- 新一代 AI 交互管道 (Day 21+ 架构)
- 新一代 AI 交互管道 (Day 28+ 架构)
大语言模型 (LLM): GLM-4.6v-Flash (通过 zai-sdk 调用智谱 AI支持多模态视觉理解)
语音识别 (ASR): SenseVoiceSmall (通过 funasr 本地部署,高精度中文识别)
语音活动检测 (VAD): Silero VAD (PyTorch 实现,含 300ms 环形缓冲,抗噪能力强)
@@ -18,6 +18,10 @@
- 计算机视觉 (CV)
核心引擎: Ultralytics YOLO (YOLO11 / YOLOv8)
模型矩阵:
- 盲道导航: yolo11l-seg-blind.engine
- 室内导盲: yolo11l-seg-indoor14.engine (14类)
- 红绿灯: yolov8n.engine
任务类型: 目标检测 (Detection) + 语义分割 (Segmentation)
推理加速: TensorRT (NVIDIA 深度学习推理引擎FP16半精度 + .engine 模型文件)
图像处理: