From: snickersun Date: Wed, 20 May 2026 08:03:46 +0000 (+0800) Subject: 拆分vc_session ,io 为ws数据收发，media为录音播放。 X-Git-Url: http://git.snicker.com.cn/?a=commitdiff_plain;h=a9cc1b2c11aee87b1fcc2fd918f688dee1387e74;p=mirrors%2FASR1603_G5.git 拆分vc_session ,io 为ws数据收发，media为录音播放。 --- diff --git a/1.doc/VC_SESSION_ARCH.md b/1.doc/VC_SESSION_ARCH.md new file mode 100644 index 00000000..7f1f9002 --- /dev/null +++ b/1.doc/VC_SESSION_ARCH.md @@ -0,0 +1,129 @@ +# VC 会话模块：目录与依赖规则 + +> 原则：**适度拆分 + 编排集中 + 子模块不互调** +> 适用：`7PIN_1_5` / `Helios_SDK` 下 `quectel/core`，MicroPython frozen（`.mpy`）。 + +**当前形态**：`vc_session.py`（编排）+ `vc_session_io.py`（WS）+ `vc_session_media.py`（采播）+ `vc_session_imports.py`（共享 import）。 + +--- + +## 1. 目录结构 + +``` +core/ +├── ai_VC.py # re-export；frozen 名不变 +├── vc_session.py # 【编排】按键/PTT、AI_VC_start/Close、定时器、FSM、__init__ +├── vc_session_io.py # 【IO】WS 建链/断链、send/recv 线程、协议 JSON 分发 +├── vc_session_media.py # 【媒体】ASR 上行/录音、TTS/播放队列、本地采播 +├── vc_session_imports.py # 子模块共享 import（仅 session/io/media） +├── vc_fsm.py # 纯 FSM +├── vc_auth / vc_buttons / vc_led / … # 外围，不 import vc_session +├── protocol.py # IOT 心跳 WS（≠ VC WS） +└── main_init.py # import ai_VC +``` + +| 文件 | 约行数 | 方法数（参考） | +|------|--------|----------------| +| `vc_session.py` | ~2250 | 编排 ~97 | +| `vc_session_io.py` | ~1730 | IO ~31 | +| `vc_session_media.py` | ~1660 | 媒体 ~76 | + +重新拆分：在 `core/` 下执行 `python _resplit_vc_session.py`（会按 `FORCE_IO` / `FORCE_MEDIA` / `FORCE_CORE` 从单体 `vc_session.py` 生成；**运行前请备份**）。 + +--- + +## 2. 依赖方向 + +```mermaid +flowchart TD + main_init --> ai_VC + ai_VC --> vc_session + vc_session --> vc_session_io + vc_session --> vc_session_media + vc_session_io --> vc_session_imports + vc_session_media --> vc_session_imports + vc_session --> vc_session_imports + vc_session_imports --> vc_fsm + vc_session_imports --> vc_common +``` + +| 规则 | 说明 | +|------|------| +| L2 编排 | 唯一 `class AI_VC` 定义处：`AI_VC(VCSessionIoMixin, VCSessionMediaMixin, Abstract)` | +| L3 io / media | **禁止互相 import**；**禁止 import vc_session** | +| 外围 `vc_*` | **禁止 import vc_session**；只 EventMesh 订阅 | + +--- + +## 3. 禁止依赖（硬约束） + +1. **`vc_session_io` ↔ `vc_session_media` 禁止互 import** +2. **io / media 禁止 import `vc_session`** +3. **io / media 禁止 `publish AI_VC_start` / `Close` 或改 `_vc_session_started`**（可 `publish` 细粒度事件如 `set_led_state`） +4. **建链、recv/send 线程仅在 io**；**录音、播放队列仅在 media** +5. **按键不得直接 `websocket_connect_VC()`** → btn 队列 → `AI_key_press` → `publish_AI_btnInfo(AI_VC_start)` +6. **线程启动失败**：`[VC] ws thread start failed`，`isconnect=0`，禁止仅打印 `ok` 无 `end` +7. **Makefile frozen**：须含 `vc_session_imports.mpy`、`vc_session_io.mpy`、`vc_session_media.mpy`、`vc_session.mpy` + +### 3.1 `import *` 与 `_thread`（易踩坑） + +`from vc_session_imports import *` **不会**导入 `vc_session_imports` 里**以下划线开头**的绑定（例如 `import _thread` 在 imports 里存在，也不会进 io/media 的全局名）。凡 mixin 内调用 `_thread.start_new_thread`，必须在 **`vc_session_io.py` / `vc_session_media.py` / `vc_session.py` 各写一行 `import _thread`**。重跑 `_resplit_vc_session.py` 后若丢行，按此补回。 + +--- + +## 4. 职责划分 + +### `vc_session.py`（编排） + +- `__init__`、EventMesh 订阅、模式/留言/会议/翻译 +- `AI_VC_start` / `Close` / `VC_init`、`check_vc_start`、idle 定时器 +- `AI_key_press` / `AI_key_release` / PTT / `ptt_begin` +- `_fsm_apply_actions`、`_vc_publish_led`、`_vc_dispatch` 的**编排侧**（实现在 io 的 recv 里调用 `self._fsm_apply_actions`） + +### `vc_session_io.py`（WS only） + +- `websocket_connect_VC` / `websocket_deconnect_VC` / `websocket_error` +- `AI_client_VC_recv`、send 队列线程、`AI_client_VC_send`、ping、Start 请求 +- `check_text` / `_vc_dispatch_server_event`（收包分发） +- `request_send_*`（经 btn 队列发 WS） + +### `vc_session_media.py`（无 connect） + +- `VC_Speech_*` / `VC_record_*`、`_uplink_*`、`request_uplink_*` +- `VC_Play_*`、`AI_audio_queue_*`、`play_tts`、留言/音乐播放回调 +- `_vc_tts_*`、语速/音色 + +--- + +## 5. 状态单写者 + +| 状态 | 写入模块 | +|------|----------| +| `AI_client_VC_isconnect`、WS 线程 id | io | +| `VC_record_state_flag`、TTS 队列 | media | +| `_vc_session_started`、`Close_flag`、`VC_satrt_flag` | 编排（`vc_session`） | +| `fsm.session` / `fsm.turn` | `vc_fsm` + `_fsm_apply_actions` | + +--- + +## 6. 发布前检查 + +- [ ] `py_compile` 通过：`vc_session.py` / `vc_session_io.py` / `vc_session_media.py` +- [ ] `vc_session_io` / `vc_session_media` / `vc_session` 顶部均有 **`import _thread`**（`import *` 不导入 `_thread`） +- [ ] io↔media 无互 import（`rg "from vc_session_" vc_session_io.py vc_session_media.py`） +- [ ] Makefile 含四个 `vc_session*.mpy` +- [ ] 开机：`websocket_connect_VC end` → `AI_VC_start3` → `rx event=started` +- [ ] 短按：`[KEY] long wake` 或 `fallback`，无 `unhandled mode branch` +- [ ] 改 `vc_auth` / `vc_buttons`：显式 import 冒烟 + +--- + +## 7. 不建议 + +- 再拆 `vc_session_ws/asr/tts` 三文件或 `key/uplink/play` 横向切片 +- io 里做 TTS 播完判定；media 里做 WS 建链 +- 合并 `protocol` IOT WS 与 VC WS + +--- + +*维护：`_resplit_vc_session.py` 中 `FORCE_IO` / `FORCE_MEDIA` / `FORCE_CORE` 调整归属后重跑脚本。* diff --git a/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/Makefile b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/Makefile index 295ff723..152fe305 100644 --- a/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/Makefile +++ b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/Makefile @@ -978,6 +978,9 @@ frozen_mpy_list = $(BUILD)/_boot.mpy \ $(BUILD)/vc_auth.mpy \ $(BUILD)/vc_ota.mpy \ $(BUILD)/vc_fsm.mpy \ + $(BUILD)/vc_session_imports.mpy \ + $(BUILD)/vc_session_io.mpy \ + $(BUILD)/vc_session_media.mpy \ $(BUILD)/vc_session.mpy \ $(BUILD)/ai_VC.mpy \ $(BUILD)/ai_alarm.mpy \ diff --git a/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/ai_VC.py b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/ai_VC.py index d1cfe0fe..3f41e1e3 100644 --- a/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/ai_VC.py +++ b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/ai_VC.py @@ -1,50 +1,50 @@ -# -*- coding: utf-8 -*- -""" -语音对话入口（兼容 frozen 模块名 ai_VC）。 -实现已拆分至 vc_* 子模块，本文件仅 re-export。 -""" -from vc_led import AI_led -from vc_hw import AI_hw_status, set_ipv4_datacall -from vc_uart import AI_UART -from vc_buttons import AI_dev, audio_buf -from vc_storage import getStorageDataManager -from vc_auth import device_res_author -from vc_ota import OTA -from vc_fsm import ( - VCSessionState, - VCTurnState, - VC_TtsRxPhase, - VCMode, - VCStateMachine, -) -from vc_session import AI_VC -from vc_utils import num_str_to_chinese -from vc_common import ( - dev_type, - VC_HEARTBEAT_LOG, - VC_PROTO_GUARDS, - VC_WS_IO_LOG, - _VC_KEEPALIVE_RX_EVENTS, - _vc_out_json_event_meta, - PRESS_AUDIO_MODE, -) - -__all__ = [ - "AI_led", - "AI_hw_status", - "set_ipv4_datacall", - "AI_UART", - "AI_dev", - "audio_buf", - "getStorageDataManager", - "device_res_author", - "OTA", - "VCSessionState", - "VCTurnState", - "VC_TtsRxPhase", - "VCMode", - "VCStateMachine", - "AI_VC", - "num_str_to_chinese", - "dev_type", -] +# -*- coding: utf-8 -*- +""" +语音对话入口（兼容 frozen 模块名 ai_VC）。 +会话：vc_session（编排）+ vc_session_io + vc_session_media；其它 vc_* 为外围。本文件仅 re-export。 +""" +from vc_led import AI_led +from vc_hw import AI_hw_status, set_ipv4_datacall +from vc_uart import AI_UART +from vc_buttons import AI_dev, audio_buf +from vc_storage import getStorageDataManager +from vc_auth import device_res_author +from vc_ota import OTA +from vc_fsm import ( + VCSessionState, + VCTurnState, + VC_TtsRxPhase, + VCMode, + VCStateMachine, +) +from vc_session import AI_VC +from vc_utils import num_str_to_chinese +from vc_common import ( + dev_type, + VC_HEARTBEAT_LOG, + VC_PROTO_GUARDS, + VC_WS_IO_LOG, + _VC_KEEPALIVE_RX_EVENTS, + _vc_out_json_event_meta, + PRESS_AUDIO_MODE, +) + +__all__ = [ + "AI_led", + "AI_hw_status", + "set_ipv4_datacall", + "AI_UART", + "AI_dev", + "audio_buf", + "getStorageDataManager", + "device_res_author", + "OTA", + "VCSessionState", + "VCTurnState", + "VC_TtsRxPhase", + "VCMode", + "VCStateMachine", + "AI_VC", + "num_str_to_chinese", + "dev_type", +] diff --git a/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_fsm.py b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_fsm.py index 80c62289..4ac24687 100644 --- a/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_fsm.py +++ b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_fsm.py @@ -1,300 +1,304 @@ -# -*- coding: utf-8 -*- -"""VC 会话状态机。""" -try: - from qp_log import qp_print as print -except Exception: - pass - -VC_record_TYPE_PCM = 0 -VC_record_TYPE_AMR = 1 -VC_play_TYPE_PCM = 0 -VC_play_TYPE_MP3 = 1 - -#websocket发送数据编号 -# AI_client_asr_num = 0 -# AI_client_chat_num = 1 -# AI_client_tts_num = 2 -AI_client_VC_num = 3 -AI_client_asr_num = 4 - - -class VCSessionState: - Disconnected = 0 - Connecting = 1 - Starting = 2 - Started = 3 - - -class VCTurnState: - Idle = 0 - Listening = 1 - SpeechStreaming = 2 - SpeechEnded = 3 - Thinking = 4 - TtsReceiving = 5 - Playing = 6 - AwaitPlayedAck = 7 - - -class VC_TtsRxPhase: - Idle = 0 - Started = 1 - ServerCompleted = 2 - - -class VCMode: - Realtime = 0 # VAD on（长对话，协议：无需 audio.create/completed） - PushToTalk = 1 # VAD off（单对话，协议：必须 audio.create/completed 包裹音频帧） - - -class VCStateMachine: - # 轻量状态机骨架（先落地骨架与动作接口，后续逐步把旧 if/flag 逻辑收敛进来）。 - # - # 约束来自《金三服务-语音对话V2.5》： - # - 收到 started 后才允许发音频帧（否则 session not initialized） - # - VAD 开启：不需要 audio.create/completed - # - VAD 关闭：必须 audio.create → frames → audio.completed - # - 播放结束 audio.played 是进入下一轮的门控信号之一（尤其关键词打断场景） - - def __init__(self, vc): - self.vc = vc - self.session = VCSessionState.Disconnected - self.turn = VCTurnState.Idle - self.last_llm_ms = 0 - self.last_tts_pending_ms = 0 - self.await_played = False - self.last_tts_completed_ms = 0 - self.tts_rx_phase = VC_TtsRxPhase.Idle - self.speech_uplink_open = 0 - - def reset_rx_phases(self): - self.tts_rx_phase = VC_TtsRxPhase.Idle - self.speech_uplink_open = 0 - - def mark_tts_rx_idle(self): - self.tts_rx_phase = VC_TtsRxPhase.Idle - - def on_server_asr_started(self): - if self.session == VCSessionState.Started: - self.speech_uplink_open = 1 - self.log("server.asr.started") - - def _turn_name(self, t): - try: - return { - VCTurnState.Idle: "Idle", - VCTurnState.Listening: "Listening", - VCTurnState.SpeechStreaming: "SpeechStreaming", - VCTurnState.SpeechEnded: "SpeechEnded", - VCTurnState.Thinking: "Thinking", - VCTurnState.TtsReceiving: "TtsReceiving", - VCTurnState.Playing: "Playing", - VCTurnState.AwaitPlayedAck: "AwaitPlayedAck", - }.get(t, str(t)) - except Exception: - return str(t) - - def _session_name(self, s): - try: - return { - VCSessionState.Disconnected: "Disconnected", - VCSessionState.Connecting: "Connecting", - VCSessionState.Starting: "Starting", - VCSessionState.Started: "Started", - }.get(s, str(s)) - except Exception: - return str(s) - - def log(self, reason=""): - # 只用于关键节点的单行状态日志，便于串口快速定位状态漂移 - try: - print("[FSM] {} session={} turn={} await_played={} tts_rx={} speech_up={}".format( - reason, - self._session_name(self.session), - self._turn_name(self.turn), - 1 if self.await_played else 0, - int(self.tts_rx_phase), - 1 if self.speech_uplink_open else 0, - )) - except Exception: - pass - - def mode(self): - try: - return VCMode.Realtime if self.vc.VC_mode == 0 else VCMode.PushToTalk - except Exception: - return VCMode.Realtime - - def on_session_start_request(self): - self.session = VCSessionState.Starting - self.turn = VCTurnState.Idle - # 新会话开始时必须清空上一轮“待播完”状态，否则会出现： - # - 切换模式/重连后 await_played 仍为 True → play_end 逻辑抢跑/不触发 → 无法自动开麦 - self.await_played = False - self.last_tts_completed_ms = 0 - self.last_llm_ms = 0 - self.reset_rx_phases() - - def on_server_started(self): - self.reset_rx_phases() - self.session = VCSessionState.Started - # 新会话 started：确保进入 Listening，且清理上轮播放门控 - self.await_played = False - self.last_tts_completed_ms = 0 - if self.turn in (VCTurnState.Idle, VCTurnState.AwaitPlayedAck): - self.turn = VCTurnState.Listening - self.log("server.started") - - def on_server_vad_silence(self): - if self.session == VCSessionState.Started: - self.turn = VCTurnState.SpeechEnded - self.log("server.vad.silence") - - def on_server_llm_created(self): - if self.session == VCSessionState.Started: - self.turn = VCTurnState.Thinking - try: - self.last_llm_ms = utime.ticks_ms() - except Exception: - self.last_llm_ms = 1 - self.log("server.llm.created") - - def on_server_tts_started(self): - if self.session == VCSessionState.Started: - self.turn = VCTurnState.TtsReceiving - self.tts_rx_phase = VC_TtsRxPhase.Started - self.log("server.tts.started") - - def on_server_tts_completed(self): - if self.session == VCSessionState.Started: - # tts.completed 只表示服务端音频下发结束，本地可能仍在播放；保持 Playing，由 await_played 门控播完上报 - self.await_played = True - self.tts_rx_phase = VC_TtsRxPhase.ServerCompleted - try: - self.last_tts_completed_ms = utime.ticks_ms() - except Exception: - self.last_tts_completed_ms = 1 - self.log("server.tts.completed") - - def on_server_canceled(self): - if self.session == VCSessionState.Started: - self.turn = VCTurnState.Listening - self.reset_rx_phases() - self.log("server.canceled") - - def on_server_completed(self): - if self.session == VCSessionState.Started: - # 本轮结束：默认回到 Listening；是否自动开麦由 action 门控决定 - # 若仍在等待本地播完（await_played），completed 不应把 turn 拉回 Listening， - # 否则会触发 completed→uplink_start 抢跑，打断播放链路。 - if not self.await_played and self.turn != VCTurnState.AwaitPlayedAck: - self.turn = VCTurnState.Listening - self.log("server.completed") - - def on_server_event(self, event, jsload_data=None): - # 事件→动作（action list）适配层。 - # 现阶段只产出“可幂等的控制动作”（uplink start/stop），避免和旧逻辑重复触发播放等副作用。 - # 后续会逐步把更多行为（play_start/audio.played/ptt create/completed）迁移到这里。 - actions = [] - try: - # LLM 阶段必须驱动 FSM.turn，否则 turn 会停在 SpeechEnded，导致： - # - _fsm_timeout_table_tick 里「Thinking 超时 / synthesis 超时」永不命中 - # - 与注释和 force_tts 前置条件不一致 - if event == "llm.created": - self.on_server_llm_created() - return actions - if event == "llm.completed": - # 若对端偶发跳过 llm.created，用 completed 把 turn 补到 Thinking，避免兜底全失效 - try: - if self.session == VCSessionState.Started: - if self.turn in (VCTurnState.SpeechEnded, VCTurnState.Listening): - self.turn = VCTurnState.Thinking - except Exception: - pass - return actions - if event == "started": - self.on_server_started() - return actions - if event == "vad.silence": - self.on_server_vad_silence() - actions.append(("uplink_stop", "vad.silence")) - return actions - if event == "tts.started": - self.on_server_tts_started() - actions.append(("uplink_stop", "tts.started")) - actions.append(("play_start", "tts.started")) - return actions - if event == "tts.completed": - self.on_server_tts_completed() - # 播放可能已在 started 时开启，这里不再强制 play_start，避免反复 open PCM - return actions - if event == "canceled": - self.on_server_canceled() - # canceled：仅长对话允许“立刻回到聆听并尝试开麦”（由 _uplink_request_start 门控保证安全幂等） - try: - if self.mode() == VCMode.Realtime and getattr(self.vc, "Close_flag", 0) == 0: - actions.append(("uplink_start", "canceled")) - except Exception: - pass - return actions - if event == "completed": - self.on_server_completed() - # completed：若不在等待 audio.played（说明本轮无 TTS），仅长对话可直接进入下一轮聆听 - try: - if self.mode() == VCMode.Realtime and getattr(self.vc, "Close_flag", 0) == 0: - if (not getattr(self, "await_played", False)) and self.turn != VCTurnState.AwaitPlayedAck: - actions.append(("uplink_start", "completed")) - except Exception: - pass - return actions - except Exception: - pass - return actions - - def on_local_audio_played(self): - # 本地播报完成后：长对话回到 Listening，等待下一轮输入 - if self.session == VCSessionState.Started: - self.turn = VCTurnState.Listening - self.await_played = False - self.last_tts_completed_ms = 0 - self.mark_tts_rx_idle() - self.log("local.audio_played") - - def on_local_play_start(self): - if self.session == VCSessionState.Started: - self.turn = VCTurnState.Playing - self.log("local.play_start") - - def on_local_synthesis_sent(self): - # synthesis 发出后到 tts.started 之间：视为 TTS 接收阶段，禁止开麦 - if self.session == VCSessionState.Started: - self.turn = VCTurnState.TtsReceiving - self.log("local.synthesis_sent") - - def on_local_play_end(self): - actions = [] - try: - if self.session != VCSessionState.Started: - return actions - if self.await_played: - actions.append(("send_audio_played", "local.play_end")) - # 长对话模式：对话过程中的 TTS 播完后，自动开麦继续当前对话。 - # 例外： - # - 刚切到长对话模式的提示音（mode_switch_long）不应自动开麦 - # - 其它提示/提醒（prompt）不应自动开麦 - # 实际开麦是否允许由 _uplink_request_start 的门控（FSM.turn / record_state 等）保证安全幂等。 - try: - if self.mode() == VCMode.Realtime and getattr(self.vc, "Close_flag", 0) == 0: - k = getattr(self.vc, "_last_tts_kind", "") or "" - if k == "dialog": - # 用户短按打断（RequestToSpeak）：等 canceled 再开麦，勿 play_end 抢跑 - if getattr(self.vc, "_vc_pending_interrupt_uplink", 0) != 1: - actions.append(("uplink_start", "local.play_end")) - except Exception: - pass - self.on_local_audio_played() - except Exception: - pass - self.log("local.play_end") - return actions - +# -*- coding: utf-8 -*- +"""VC 会话状态机。""" +try: + from qp_log import qp_print as print +except Exception: + pass + +VC_record_TYPE_PCM = 0 +VC_record_TYPE_AMR = 1 +VC_play_TYPE_PCM = 0 +VC_play_TYPE_MP3 = 1 + +#websocket发送数据编号 +# AI_client_asr_num = 0 +# AI_client_chat_num = 1 +# AI_client_tts_num = 2 +AI_client_VC_num = 3 +AI_client_asr_num = 4 + + +class VCSessionState: + Disconnected = 0 + Connecting = 1 + Starting = 2 + Started = 3 + + +class VCTurnState: + Idle = 0 + Listening = 1 + SpeechStreaming = 2 + SpeechEnded = 3 + Thinking = 4 + TtsReceiving = 5 + Playing = 6 + AwaitPlayedAck = 7 + + +class VC_TtsRxPhase: + Idle = 0 + Started = 1 + ServerCompleted = 2 + + +class VCMode: + Realtime = 0 # VAD on（长对话，协议：无需 audio.create/completed） + PushToTalk = 1 # VAD off（单对话，协议：必须 audio.create/completed 包裹音频帧） + + +class VCStateMachine: + # 轻量状态机骨架（先落地骨架与动作接口，后续逐步把旧 if/flag 逻辑收敛进来）。 + # + # 约束来自《金三服务-语音对话V2.5》： + # - 收到 started 后才允许发音频帧（否则 session not initialized） + # - VAD 开启：不需要 audio.create/completed + # - VAD 关闭：必须 audio.create → frames → audio.completed + # - 播放结束 audio.played 是进入下一轮的门控信号之一（尤其关键词打断场景） + + def __init__(self, vc): + self.vc = vc + self.session = VCSessionState.Disconnected + self.turn = VCTurnState.Idle + self.last_llm_ms = 0 + self.last_tts_pending_ms = 0 + self.await_played = False + self.last_tts_completed_ms = 0 + self.tts_rx_phase = VC_TtsRxPhase.Idle + self.speech_uplink_open = 0 + + def reset_rx_phases(self): + self.tts_rx_phase = VC_TtsRxPhase.Idle + self.speech_uplink_open = 0 + + def mark_tts_rx_idle(self): + self.tts_rx_phase = VC_TtsRxPhase.Idle + + def on_server_asr_started(self): + if self.session == VCSessionState.Started: + self.speech_uplink_open = 1 + self.log("server.asr.started") + + def _turn_name(self, t): + try: + return { + VCTurnState.Idle: "Idle", + VCTurnState.Listening: "Listening", + VCTurnState.SpeechStreaming: "SpeechStreaming", + VCTurnState.SpeechEnded: "SpeechEnded", + VCTurnState.Thinking: "Thinking", + VCTurnState.TtsReceiving: "TtsReceiving", + VCTurnState.Playing: "Playing", + VCTurnState.AwaitPlayedAck: "AwaitPlayedAck", + }.get(t, str(t)) + except Exception: + return str(t) + + def _session_name(self, s): + try: + return { + VCSessionState.Disconnected: "Disconnected", + VCSessionState.Connecting: "Connecting", + VCSessionState.Starting: "Starting", + VCSessionState.Started: "Started", + }.get(s, str(s)) + except Exception: + return str(s) + + def log(self, reason=""): + # 只用于关键节点的单行状态日志，便于串口快速定位状态漂移 + try: + print("[FSM] {} session={} turn={} await_played={} tts_rx={} speech_up={}".format( + reason, + self._session_name(self.session), + self._turn_name(self.turn), + 1 if self.await_played else 0, + int(self.tts_rx_phase), + 1 if self.speech_uplink_open else 0, + )) + except Exception: + pass + + def mode(self): + try: + return VCMode.Realtime if self.vc.VC_mode == 0 else VCMode.PushToTalk + except Exception: + return VCMode.Realtime + + def on_session_start_request(self): + self.session = VCSessionState.Starting + self.turn = VCTurnState.Idle + # 新会话开始时必须清空上一轮“待播完”状态，否则会出现： + # - 切换模式/重连后 await_played 仍为 True → play_end 逻辑抢跑/不触发 → 无法自动开麦 + self.await_played = False + self.last_tts_completed_ms = 0 + self.last_llm_ms = 0 + self.reset_rx_phases() + + def on_server_started(self): + self.reset_rx_phases() + self.session = VCSessionState.Started + # 新会话 started：确保进入 Listening，且清理上轮播放门控 + self.await_played = False + self.last_tts_completed_ms = 0 + if self.turn in (VCTurnState.Idle, VCTurnState.AwaitPlayedAck): + self.turn = VCTurnState.Listening + self.log("server.started") + + def on_server_vad_silence(self): + if self.session == VCSessionState.Started: + self.turn = VCTurnState.SpeechEnded + self.log("server.vad.silence") + + def on_server_llm_created(self): + if self.session == VCSessionState.Started: + self.turn = VCTurnState.Thinking + try: + self.last_llm_ms = utime.ticks_ms() + except Exception: + self.last_llm_ms = 1 + self.log("server.llm.created") + + def on_server_tts_started(self): + if self.session == VCSessionState.Started: + self.turn = VCTurnState.TtsReceiving + self.tts_rx_phase = VC_TtsRxPhase.Started + # 新一轮 TTS：清上轮 completed 时间戳，避免 play_end 用陈旧 since_done 触发 too_long 截断 + self.last_tts_completed_ms = 0 + self.log("server.tts.started") + + def on_server_tts_completed(self): + if self.session == VCSessionState.Started: + # tts.completed 只表示服务端音频下发结束，本地可能仍在播放；保持 Playing，由 await_played 门控播完上报 + self.await_played = True + self.tts_rx_phase = VC_TtsRxPhase.ServerCompleted + try: + self.last_tts_completed_ms = utime.ticks_ms() + except Exception: + # 勿置 1：ticks_diff(now,1) 会误判为超时并过早 VC_Play_End + self.last_tts_completed_ms = 0 + self.log("server.tts.completed") + + def on_server_canceled(self): + if self.session == VCSessionState.Started: + self.turn = VCTurnState.Listening + self.reset_rx_phases() + self.log("server.canceled") + + def on_server_completed(self): + if self.session == VCSessionState.Started: + # 本轮结束：默认回到 Listening；是否自动开麦由 action 门控决定 + # 若仍在等待本地播完（await_played），completed 不应把 turn 拉回 Listening， + # 否则会触发 completed→uplink_start 抢跑，打断播放链路。 + if not self.await_played and self.turn != VCTurnState.AwaitPlayedAck: + self.turn = VCTurnState.Listening + self.log("server.completed") + + def on_server_event(self, event, jsload_data=None): + # 事件→动作（action list）适配层。 + # 现阶段只产出“可幂等的控制动作”（uplink start/stop），避免和旧逻辑重复触发播放等副作用。 + # 后续会逐步把更多行为（play_start/audio.played/ptt create/completed）迁移到这里。 + actions = [] + try: + # LLM 阶段必须驱动 FSM.turn，否则 turn 会停在 SpeechEnded，导致： + # - _fsm_timeout_table_tick 里「Thinking 超时 / synthesis 超时」永不命中 + # - 与注释和 force_tts 前置条件不一致 + if event == "llm.created": + self.on_server_llm_created() + return actions + if event == "llm.completed": + # 若对端偶发跳过 llm.created，用 completed 把 turn 补到 Thinking，避免兜底全失效 + try: + if self.session == VCSessionState.Started: + if self.turn in (VCTurnState.SpeechEnded, VCTurnState.Listening): + self.turn = VCTurnState.Thinking + except Exception: + pass + return actions + if event == "started": + self.on_server_started() + return actions + if event == "vad.silence": + self.on_server_vad_silence() + actions.append(("uplink_stop", "vad.silence")) + return actions + if event == "tts.started": + self.on_server_tts_started() + actions.append(("uplink_stop", "tts.started")) + actions.append(("play_start", "tts.started")) + return actions + if event == "tts.completed": + self.on_server_tts_completed() + # 播放可能已在 started 时开启，这里不再强制 play_start，避免反复 open PCM + return actions + if event == "canceled": + self.on_server_canceled() + # canceled：仅长对话允许“立刻回到聆听并尝试开麦”（由 _uplink_request_start 门控保证安全幂等） + try: + if self.mode() == VCMode.Realtime and getattr(self.vc, "Close_flag", 0) == 0: + actions.append(("uplink_start", "canceled")) + except Exception: + pass + return actions + if event == "completed": + self.on_server_completed() + # completed：若不在等待 audio.played（说明本轮无 TTS），仅长对话可直接进入下一轮聆听 + try: + if self.mode() == VCMode.Realtime and getattr(self.vc, "Close_flag", 0) == 0: + if (not getattr(self, "await_played", False)) and self.turn != VCTurnState.AwaitPlayedAck: + actions.append(("uplink_start", "completed")) + except Exception: + pass + return actions + except Exception: + pass + return actions + + def on_local_audio_played(self): + # 本地播报完成后：长对话回到 Listening，等待下一轮输入 + if self.session == VCSessionState.Started: + self.turn = VCTurnState.Listening + self.await_played = False + self.last_tts_completed_ms = 0 + self.mark_tts_rx_idle() + self.log("local.audio_played") + + def on_local_play_start(self): + if self.session == VCSessionState.Started: + self.turn = VCTurnState.Playing + self.log("local.play_start") + + def on_local_synthesis_sent(self): + # synthesis 发出后到 tts.started 之间：视为 TTS 接收阶段，禁止开麦 + if self.session == VCSessionState.Started: + self.turn = VCTurnState.TtsReceiving + self.log("local.synthesis_sent") + + def on_local_play_end(self): + actions = [] + try: + if self.session != VCSessionState.Started: + return actions + if self.await_played: + actions.append(("send_audio_played", "local.play_end")) + # 长对话模式：对话过程中的 TTS 播完后，自动开麦继续当前对话。 + # 例外： + # - 刚切到长对话模式的提示音（mode_switch_long）不应自动开麦 + # - 其它提示/提醒（prompt）不应自动开麦 + # - boot_prompt：started 阶段已跳过开麦，播完需自动进入首轮聆听（155536） + # 实际开麦是否允许由 _uplink_request_start 的门控（FSM.turn / record_state 等）保证安全幂等。 + try: + if self.mode() == VCMode.Realtime and getattr(self.vc, "Close_flag", 0) == 0: + k = getattr(self.vc, "_last_tts_kind", "") or "" + if k in ("dialog", "boot_prompt"): + # 用户短按打断（RequestToSpeak）：等 canceled 再开麦，勿 play_end 抢跑 + if getattr(self.vc, "_vc_pending_interrupt_uplink", 0) != 1: + actions.append(("uplink_start", "local.play_end")) + except Exception: + pass + self.on_local_audio_played() + except Exception: + pass + self.log("local.play_end") + return actions + diff --git a/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_session.py b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_session.py index a9a34418..c8cf361b 100644 --- a/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_session.py +++ b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_session.py @@ -1,5886 +1,2264 @@ -# -*- coding: utf-8 -*- -"""语音对话 WebSocket / ASR / TTS 主业务。""" -from common import Abstract -try: - from qp_log import qp_print as print -except Exception: - pass -import EventMesh -import uwebsocket -import usys -import gc -import urandom -import uhashlib -import ubinascii -import ujson -import osTimer -import _thread -from queue import Queue -import utime -import audio -import uos -from machine import Pin -import modem -import sim -import net -import base64 -import request -import ql_fs -from misc import Power -import ntptime -import machine -import dataCall -import constants -import versions -import version_util -try: - import vc_config -except Exception: - vc_config = None -try: - import vc_text_intent -except Exception: - vc_text_intent = None -try: - import VAD -except Exception: - pass -try: - import usr.ai_alarm as _ai_alarm -except Exception: - import ai_alarm as _ai_alarm -from vc_common import ( - dev_type, - VC_HEARTBEAT_LOG, - VC_PROTO_GUARDS, - VC_WS_IO_LOG, - _VC_KEEPALIVE_RX_EVENTS, - _vc_out_json_event_meta, - PRESS_AUDIO_MODE, -) -from vc_fsm import ( - VCSessionState, - VCTurnState, - VC_TtsRxPhase, - VCMode, - VCStateMachine, - VC_record_TYPE_PCM, - VC_record_TYPE_AMR, - VC_play_TYPE_PCM, - VC_play_TYPE_MP3, - AI_client_VC_num, - AI_client_asr_num, -) -from vc_ota import OTA -from vc_utils import num_str_to_chinese - -class AI_VC(Abstract): - def get_vc_tts_playback_active(self, *args, **kwargs): - try: - return 1 if self._vc_tts_playback_active() else 0 - except Exception: - return 0 - - def get_vc_stream_active(self, *args, **kwargs): - # 供本地提示音侧查询：TTS 流播放期间本地不可抢占。 - try: - if self._vc_tts_playback_active(): - return 1 - except Exception: - pass - try: - if getattr(self.fsm, "await_played", False): - return 1 - except Exception: - pass - try: - if getattr(self.fsm, "tts_rx_phase", VC_TtsRxPhase.Idle) != VC_TtsRxPhase.Idle: - return 1 - except Exception: - pass - try: - t = getattr(self.fsm, "turn", VCTurnState.Idle) - if t in (VCTurnState.TtsReceiving, VCTurnState.Playing, VCTurnState.AwaitPlayedAck): - return 1 - except Exception: - pass - try: - if getattr(self, "_tts_pending", 0) == 1: - return 1 - except Exception: - pass - return 0 - - def _vc_fsm_log_snip(self): - try: - f = self.fsm - return "sess={} turn={} tts_rx={} await={} sp_up={}".format( - int(getattr(f, "session", -1)), - int(getattr(f, "turn", -1)), - int(getattr(f, "tts_rx_phase", -1)), - 1 if getattr(f, "await_played", False) else 0, - 1 if getattr(f, "speech_uplink_open", 0) else 0, - ) - except Exception: - return "fsm=?" - - def _vc_tts_pipeline_busy(self): - try: - if getattr(self.fsm, "await_played", False): - return True - if getattr(self.fsm, "tts_rx_phase", VC_TtsRxPhase.Idle) != VC_TtsRxPhase.Idle: - return True - t = getattr(self.fsm, "turn", VCTurnState.Idle) - if t in (VCTurnState.TtsReceiving, VCTurnState.Playing, VCTurnState.AwaitPlayedAck): - return True - if getattr(self, "_tts_pending", 0) == 1: - return True - if getattr(self, "VC_Play_flag", 0) == 1: - return True - if self._vc_local_play_busy(): - return True - except Exception: - pass - return False - - def _vc_local_play_busy(self): - try: - if getattr(self, "_vc_local_play_active", 0) != 1: - return False - until = getattr(self, "_vc_local_play_until_ms", 0) - if until and utime.ticks_diff(until, utime.ticks_ms()) > 0: - return True - self._vc_local_play_active = 0 - self._vc_local_play_until_ms = 0 - except Exception: - pass - return False - - def _vc_capture_allowed(self, reason=""): - """采集/上行开麦前：下行播放或本地短音未结束则禁止。""" - try: - if self._vc_tts_pipeline_busy(): - return False - if self._vc_local_play_busy(): - return False - except Exception: - return False - return True - - def _vc_stop_downlink_for_interrupt(self, reason="interrupt"): - """详设 DES-SCN-003：打断/取消时先本地停播清队列，再允许开麦。""" - try: - if getattr(self, "AI_audio_queue", None) is not None: - if self.AI_audio_queue.empty() != True: - self.AI_audio_queue.clear() - except Exception: - pass - try: - if self._vc_tts_pipeline_busy() or getattr(self, "is_ruppt_tts", 0) == 1: - self._drop_tts_audio = 1 - except Exception: - pass - try: - self._ptt_hard_mute_local_playback() - except Exception: - pass - try: - if getattr(self, "VC_Play_flag", 0) == 1 or getattr(self, "play_pcm", None) is not None: - self.VC_Play_End() - except Exception: - pass - try: - self._vc_local_play_active = 0 - self._vc_local_play_until_ms = 0 - except Exception: - pass - try: - self.fsm.await_played = False - self._need_audio_played_ack = 0 - self._tts_pending = 0 - self._tts_pending_ms = 0 - except Exception: - pass - try: - pending_intr = getattr(self, "_vc_pending_interrupt_uplink", 0) == 1 - if ( - not pending_intr - and getattr(self, "VC_record_state_flag", 0) != 0 - and getattr(self, "record_data_save_flag", 0) == 0 - ): - self._uplink_request_stop("stop_downlink/" + str(reason)) - except Exception: - pass - - def _vc_local_play(self, path, play_mode=1, loop=1, hold_ms=650): - """本地 MP3 短音：播前停录/停流式 TTS，避免与麦克风采集叠用。""" - try: - if getattr(self, "record_data_save_flag", 0) == 0 and getattr(self, "VC_record_state_flag", 0) != 0: - self._uplink_request_stop("local_play") - utime.sleep_ms(30) - except Exception: - pass - try: - if self._vc_tts_playback_active(): - self._ptt_hard_mute_local_playback() - try: - self.VC_Play_End() - except Exception: - pass - except Exception: - pass - try: - hold = int(hold_ms) - if hold < 200: - hold = 200 - self._vc_local_play_active = 1 - self._vc_local_play_until_ms = utime.ticks_add(utime.ticks_ms(), hold) - except Exception: - pass - try: - self.aud.play(play_mode, loop, path) - except Exception as e: - try: - self._vc_local_play_active = 0 - self._vc_local_play_until_ms = 0 - except Exception: - pass - usys.print_exception(e) - - def _vc_tts_playback_active(self): - """仅服务端 TTS/PCM 流在播（VC_Play_Start）；aud.play 本地短音不算。""" - try: - if getattr(self, "VC_Play_flag", 0) == 1: - return True - except Exception: - pass - try: - if getattr(self, "play_pcm", None) is not None: - return True - except Exception: - pass - return False - - def _vc_vad_uplink_stop_eligible(self): - # 对应旧 VC_state >= SpeechStart - try: - if getattr(self.fsm, "speech_uplink_open", 0): - return True - t = getattr(self.fsm, "turn", VCTurnState.Idle) - return t not in (VCTurnState.Idle, VCTurnState.Listening) - except Exception: - return False - - def _vc_long_idle_active_speech_or_later(self): - # 对应旧 VC_state >= SpeechIng - try: - t = getattr(self.fsm, "turn", VCTurnState.Idle) - return t not in (VCTurnState.Idle, VCTurnState.Listening) - except Exception: - return False - - def _vc_is_business_idle(self): - # Level2：LED ai_state 为 idle，且无录音/流播/音乐闹钟等（与 WS ping 无关） - try: - if getattr(self, "music_play_flag", 0) == 1: - return False - if getattr(self, "alarm_play_flag", 0) == 1: - return False - except Exception: - pass - try: - if getattr(self, "VC_record_state_flag", 0) != 0: - return False - if getattr(self, "VC_Play_flag", 0) != 0: - return False - except Exception: - pass - try: - if EventMesh.publish("get_vc_stream_active") == 1: - return False - except Exception: - pass - try: - st = EventMesh.publish("get_led_debug_state") - if st and isinstance(st, dict): - if (st.get("ai") or "") != "idle": - return False - except Exception: - return False - return True - - def _vc_idle_deep_sleep_impl(self): - try: - _need_close = (self.VC_satrt_flag == 1) or (getattr(self, "AI_client_VC_isconnect", 0) == 1) - print("[VC][idle_L2] 触发休眠流程 need_Close={} VC_mode={}".format( - 1 if _need_close else 0, getattr(self, "VC_mode", -1))) - except Exception: - pass - try: - if self.VC_satrt_flag == 1 or getattr(self, "AI_client_VC_isconnect", 0) == 1: - try: - print("[VC][idle_L2] 调用 Close 断开会话/链路") - except Exception: - pass - self.Close() - except Exception: - pass - try: - utime.sleep_ms(100) - except Exception: - pass - try: - print("[VC][idle_L2] 发布 Hsensor_set_sleep") - EventMesh.publish("Hsensor_set_sleep") - except Exception: - pass - try: - print("[VC][idle_L2] 发布 SYSTEM_SLEEP ({})".format(constants.EVENT.SYSTEM_SLEEP)) - EventMesh.publish(constants.EVENT.SYSTEM_SLEEP) - except Exception: - pass - - def _vc_rx_resets_long_idle_timer(self, recv_data): - """是否应因本帧下行清零 close_timerout（排除保活/心跳，含 V2.5 文本 ping 对端应答）。""" - try: - if recv_data is None: - return False - if isinstance(recv_data, (bytes, bytearray)): - return True - if not isinstance(recv_data, str): - return False - t = recv_data.strip() - if not t: - return False - tl = t.lower() - if tl in ("ping", "pong"): - return False - try: - jd = ujson.loads(t) - except Exception: - return True - ev = jd.get("event") - if ev is None or ev == "": - ev = jd.get("type") - if ev is None: - return True - if str(ev).strip().lower() in _VC_KEEPALIVE_RX_EVENTS: - return False - return True - except Exception: - return True - - # ------------------------------------------------------------------------- - # 电源键 / AI 键：按 VC_mode 拆函数，便于阅读与修改（长对话 vs 单对话 PTT） - # ------------------------------------------------------------------------- - def _handle_pwk_short_release(self, event=None, msg=None, merged_ai_tail=False): - """电源键短按：读留言标志、退会议/翻译；否则 Close，或单键模式下走 AI_key_release。""" - print("pwk_key_release") - if EventMesh.publish("get_recv_voice") == 1: - EventMesh.publish("update_play_new_voice_info") - self.recv_voice = 0 - EventMesh.publish("set_history_voice_flag", 0) - self.AI_new_voice_url = "" - if self.trans_flag == 1 or self.meet_flag == 1: - EventMesh.publish("exit_meet_mode") - EventMesh.publish("exit_trans_mode") - return - if merged_ai_tail: - self.AI_key_release(event, msg) - else: - EventMesh.publish("publish_AI_btnInfo", {"func": self.Close}) - EventMesh.publish("stop_sos_call") - - def _ai_key_press_ptt_mode(self): - """单对话 PTT：短按仅停 TTS（不换 sid）；非播放态短按吞掉；长按由 AI_key_long_press→ptt_begin。""" - if self.VC_mode != 1: - return False - try: - playing = False - try: - t = getattr(self.fsm, "turn", VCTurnState.Idle) - playing = t in (VCTurnState.TtsReceiving, VCTurnState.Playing) - except Exception: - playing = False - if playing or self._vc_tts_pipeline_busy(): - self._ptt_short_press_stop = 1 - try: - self._ptt_hard_mute_local_playback() - except Exception: - pass - try: - self.VC_Play_End() - except Exception: - pass - try: - self.fsm.await_played = False - except Exception: - pass - try: - self._ptt_led_lock_listening = 0 - self._ptt_interrupt_active = 0 - self.is_ruppt_tts = 0 - except Exception: - pass - try: - self._vc_publish_led("idle", "ptt_short_press/stop_tts") - except Exception: - pass - return True - except Exception: - pass - return True - - def _ai_key_press_long_dialogue_interrupt(self): - """长对话：TTS/播放忙或 turn==Thinking 时短按 → RequestToSpeak（先 cancel，canceled 后再开麦）。""" - if self.VC_mode != 0: - return False - try: - if getattr(self, "AI_client_VC_isconnect", 0) != 1 or getattr(self, "_vc_session_started", 0) != 1: - return False - except Exception: - return False - need_cancel = False - if self._vc_tts_pipeline_busy(): - need_cancel = True - else: - try: - if getattr(self.fsm, "turn", VCTurnState.Idle) == VCTurnState.Thinking: - need_cancel = True - except Exception: - pass - if not need_cancel: - return False - print("[KEY] long interrupt -> RequestToSpeak (cancel then uplink)") - self.is_ruppt_tts = 1 - try: - self._vc_pending_interrupt_uplink = 1 - except Exception: - pass - EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_RequestToSpeak}) - return True - - def _ai_key_press_long_dialogue_wake(self): - """长对话未连接或未 started：直接 AI_VC_start，勿 Close+sleep。""" - if self.VC_mode != 0: - return False - if getattr(self, "voice_flag", 0) == 1 or getattr(self, "meet_flag", 0) == 1 or getattr(self, "trans_flag", 0) == 1: - return False - try: - if getattr(self, "AI_client_VC_isconnect", 0) == 1 and getattr(self, "_vc_session_started", 0) == 1: - return False - except Exception: - pass - print("[KEY] long wake AI_VC_start (no Close)") - try: - self.Close_flag = 0 - except Exception: - pass - self.get_config_timer_count = 0 - self.key_state = 1 - self.AI_TTS_str = "" - self.AI_ASR_str = "" - self.AI_ASR_str_tmp = "" - self.AI_VC_start() - self.AI_key_long_press_flag = 0 - return True - - def _vc_led_alert_rb_flash(self): - try: - EventMesh.publish("set_led_state", {"alert": "rb_flash_2"}) - except Exception: - pass - - def _vc_clear_stale_record_state(self, reason=""): - """record_state_flag 非 0 但可能无真实录音时先停录复位，避免 uplink 门控拒开麦。""" - try: - if getattr(self, "VC_record_state_flag", 0) != 0: - self.VC_Speech_End() - except Exception: - try: - self.VC_record_state_flag = 0 - except Exception: - pass - - def _ai_key_press_long_dialogue_when_session_ready(self): - """长对话已连接且 session å·² started：非播放态直接开麦续聊（勿 idle 时发 cancel）。""" - if self.VC_mode != 0: - return False - try: - if getattr(self, "AI_client_VC_isconnect", 0) == 1 and getattr(self, "_vc_session_started", 0) == 1: - # 模式切换提示音会置 Close_flag=1；用户再短按须恢复可开麦 - try: - self.Close_flag = 0 - except Exception: - pass - self._vc_clear_stale_record_state("key.press/continue") - self.request_uplink_start("key.press/continue") - return True - except Exception: - pass - return False - - def _ai_key_release_handle_ptt(self): - """单对话 PTT 松手：仅在上行录音中时 end_speech；禁止误触发 AI_VC_start（否则新 sid / cancel 竞态）。""" - if self.AI_key_long_press_flag == 1: - if getattr(self, "VC_record_state_flag", 0) != 0: - try: - gap = 200 - if "vc_config" in globals() and vc_config: - gap = int(vc_config.get_int("PTT_RELEASE_END_SPEECH_DELAY_MS", 200)) - if gap > 0: - utime.sleep_ms(gap) - except Exception: - pass - print("[PTT] release end_speech asr_str={!r} rec={}".format( - self.AI_ASR_str, getattr(self, "VC_record_state_flag", -1))) - self.AI_client_VC_end_speech() - else: - self.AI_key_long_press_flag = 0 - try: - if getattr(self, "_ptt_short_press_stop", 0) == 1: - self._ptt_short_press_stop = 0 - except Exception: - pass - print("[PTT] release_no_uplink_skip") - return - try: - if getattr(self, "_ptt_short_press_stop", 0) == 1: - self._ptt_short_press_stop = 0 - except Exception: - pass - print("[PTT] release_short_no_long") - - def _ai_key_long_press_route(self, event=None, msg=None): - """长按：翻译先 Close 再开翻译录音；单对话→ptt_begin；长对话→与短按一致走 AI_key_press 链路。""" - if self.trans_flag == 1: - self.Close() - EventMesh.publish("start_voice_record", 4) - self.AI_key_long_press_flag = 1 - if self.VC_mode == 1: - EventMesh.publish("publish_AI_btnInfo", {"func": self.ptt_begin}) - else: - EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_key_press}) - - # ===== Push-To-Talk（单对话模式）===== - # 语义： - # - 长按：无论当前处于 TTS/Thinking/任何中间态，立即打断并进入新一轮录音上行 - # - 松开：停止录音并等待服务端返回（由 AI_client_VC_end_speech 触发） - def _ptt_end_led_suppress(self): - # 保持 _vc_play_end_suppress_idle_led / _ptt_led_lock_listening 直至真开录（VC_Speech_Start） - pass - - def _ptt_should_drop_inbound_tts(self): - """单对话打断窗口：丢弃迟到 TTS 帧 / 禁止起播。""" - try: - if getattr(self, "VC_mode", -1) != 1: - return False - if getattr(self, "_drop_tts_audio", 0) == 1: - return True - if getattr(self, "is_ruppt_tts", 0) == 1: - return True - if getattr(self, "_ptt_interrupt_active", 0) == 1: - return True - if getattr(self, "_vc_serial_canceling", 0) == 1: - return True - if getattr(self, "_ptt_pending_arm", 0) == 1: - return True - except Exception: - pass - return False - - def _ptt_should_skip_remote_playback(self): - return self._ptt_should_drop_inbound_tts() - - def _ptt_enter_interrupt_arm(self): - try: - self._ptt_arm_epoch = int(getattr(self, "_ptt_arm_epoch", 0)) + 1 - self._ptt_interrupt_active = 1 - self._ptt_led_lock_listening = 1 - self._vc_play_end_suppress_idle_led = 1 - self._drop_tts_audio = 1 - self.is_ruppt_tts = 1 - except Exception: - pass - - def _ptt_abort_interrupt_arm(self): - try: - self._ptt_interrupt_active = 0 - self._ptt_led_lock_listening = 0 - self._vc_play_end_suppress_idle_led = 0 - self._drop_tts_audio = 0 - self.is_ruppt_tts = 0 - except Exception: - pass - - def _ptt_on_record_hardware_started(self): - """真开录后：允许新 TTS、放开灯锁；旧 TTS 已在打断期丢弃。""" - try: - self._ptt_interrupt_active = 0 - self._ptt_led_lock_listening = 0 - self._vc_play_end_suppress_idle_led = 0 - self._drop_tts_audio = 0 - self.is_ruppt_tts = 0 - except Exception: - pass - - def _ptt_hard_mute_local_playback(self): - try: - self.VC_Play_flag = 0 - if getattr(self, "VC_play_TYPE", None) == VC_play_TYPE_PCM: - if getattr(self, "play_pcm", None) is not None: - try: - self.play_pcm.stream_stop() - self.play_pcm.close() - except Exception: - pass - self.play_pcm = None - try: - self.aud.stopPlayStream() - self.aud.stopAll() - except Exception: - pass - if getattr(self, "AI_audio_queue", None) is not None: - try: - if self.AI_audio_queue.empty() != True: - self.AI_audio_queue.clear() - except Exception: - pass - except Exception: - pass - try: - gap = 0 - if "vc_config" in globals() and vc_config: - gap = int(vc_config.get_int("PTT_POST_MUTE_MS", 30)) - if gap > 0: - utime.sleep_ms(gap) - except Exception: - pass - - def battery_priority_mute_vc(self, event=None, msg=None): - """低电/关机提示音：短暂停 VC TTS，避免被流式播放挡住。""" - try: - self._ptt_hard_mute_local_playback() - except Exception: - pass - try: - if getattr(self, "VC_Play_flag", 0) == 1 or getattr(self, "play_pcm", None) is not None: - self.VC_Play_End() - except Exception: - pass - - def _vc_enqueue_btn(self, func, data=None): - """统一经 publish_AI_btnInfo 入队，在 btnInfo_queue_thread 串行执行。""" - try: - if data is not None: - EventMesh.publish("publish_AI_btnInfo", {"func": func, "data": data}) - else: - EventMesh.publish("publish_AI_btnInfo", {"func": func}) - except Exception: - pass - - def _vc_ptt_cancel_wait_ms(self): - try: - if "vc_config" in globals() and vc_config: - return int(vc_config.get_int("PTT_CANCEL_WAIT_MS", 2500)) - except Exception: - pass - return 2500 - - def _vc_ptt_need_interrupt_wait(self): - """单对话：服务端仍在思考/收 TTS/播报时，开录前须先 cancel 并等 canceled。""" - try: - if getattr(self, "VC_mode", -1) != 1: - return False - turn = getattr(self.fsm, "turn", VCTurnState.Idle) - if self._vc_tts_pipeline_busy(): - return True - if turn in ( - VCTurnState.Thinking, VCTurnState.TtsReceiving, - VCTurnState.Playing, VCTurnState.AwaitPlayedAck, - ): - return True - except Exception: - pass - return False - - def _vc_begin_ptt_cancel_wait(self, reason="ptt_interrupt"): - try: - self._ptt_pending_arm = 1 - self._vc_serial_canceling = 1 - try: - self._vc_cancel_deadline_ms = utime.ticks_add( - utime.ticks_ms(), self._vc_ptt_cancel_wait_ms()) - except Exception: - self._vc_cancel_deadline_ms = 0 - json_data = {"event": "cancel", "payload": {}} - self.AI_client_VC_send(ujson.dumps(json_data)) - try: - self.fsm.reset_rx_phases() - except Exception: - pass - self._drop_tts_audio = 1 - print("[VC][serial] cancel_wait reason={} turn={}".format( - reason, getattr(getattr(self, "fsm", None), "turn", "?"))) - except Exception: - pass - - def _vc_on_ptt_cancel_settled(self, via="canceled"): - """btn 线程：cancel 已收尾，执行挂起的 PTT 开录。""" - try: - if not getattr(self, "_vc_serial_canceling", 0) and not getattr(self, "_ptt_pending_arm", 0): - return - self._vc_serial_canceling = 0 - self._vc_cancel_deadline_ms = 0 - if getattr(self, "_ptt_pending_arm", 0): - self._ptt_pending_arm = 0 - print("[VC][serial] ptt_arm after {}".format(via)) - self._ptt_arm_uplink() - except Exception: - pass - - def _vc_tick_serial_cancel_timeout(self): - if not getattr(self, "_vc_serial_canceling", 0): - return - try: - dl = getattr(self, "_vc_cancel_deadline_ms", 0) - if dl and utime.ticks_diff(utime.ticks_ms(), dl) >= 0: - print("[VC][serial] cancel_wait timeout -> arm") - self._vc_on_ptt_cancel_settled(via="timeout") - except Exception: - pass - - def _vc_fsm_dispatch_cb(self, data): - """btn 线程：处理服务端 FSM 事件（单对话串行）。""" - try: - ev, jsload = data - except Exception: - return - try: - actions = self.fsm.on_server_event(ev, jsload) - self._fsm_apply_actions(actions) - except Exception as e: - usys.print_exception(e) - if ev == "canceled": - self._vc_on_ptt_cancel_settled(via="canceled") - - def _vc_dispatch_server_event(self, ev, jsload_data=None): - try: - if ev == "canceled": - self._vc_stop_downlink_for_interrupt("canceled") - except Exception: - pass - if getattr(self, "VC_mode", -1) == 1: - self._vc_enqueue_btn(self._vc_fsm_dispatch_cb, (ev, jsload_data)) - else: - try: - self._fsm_apply_actions(self.fsm.on_server_event(ev, jsload_data)) - except Exception: - pass - - def _ptt_arm_uplink(self, t0=0): - """btn 线程：cancel 已结束或无需 cancel 时，发 audio.create 并开麦。""" - try: - self.is_ruppt_tts = 0 - except Exception: - pass - try: - self._ptt_audio_created = 0 - except Exception: - pass - try: - self.AI_ASR_flag = 0 - except Exception: - pass - try: - self.AI_client_VC_send_speech() - except Exception: - pass - try: - self.VC_Speech_Start_now() - except Exception: - pass - try: - self._vc_publish_led("listening", "ptt_begin/recording") - except Exception: - pass - self._ptt_end_led_suppress() - try: - if t0: - print("[PTT] record started (ms={})".format(utime.ticks_ms() - t0)) - except Exception: - pass - - def ptt_begin(self, reason="key.long_press"): - try: - if self.VC_mode != 1: - return - except Exception: - return - - # PTT 新一轮开始：清理模式切换/提示音退出标志 - try: - self.Close_flag = 0 - self._ptt_await_server = 0 - except Exception: - pass - - t0 = 0 - try: - t0 = utime.ticks_ms() - except Exception: - t0 = 0 - - self._ptt_enter_interrupt_arm() - try: - self._vc_publish_led("listening", "ptt_begin/pre") - except Exception: - pass - - # 本地先硬静音再 VC_init，避免 DMA/队列尾包漏播 - try: - self._ptt_hard_mute_local_playback() - except Exception: - pass - try: - self.VC_init() - except Exception: - pass - try: - if getattr(self, "VC_Play_flag", 0) == 1 or getattr(self, "play_pcm", None) is not None: - self.VC_Play_End() - except Exception: - pass - try: - self._ptt_hard_mute_local_playback() - except Exception: - pass - - # 若上一轮录音尚未结束，先强制停掉，避免 record_thread 复用出错 - try: - if getattr(self, "VC_record_state_flag", 0) != 0: - self.VC_Speech_End() - except Exception: - pass - - # 已在等 canceled：只刷新「待开录」，勿重复 audio.create - try: - if getattr(self, "_vc_serial_canceling", 0): - self._ptt_pending_arm = 1 - print("[PTT] cancel in progress, refresh pending arm") - self._ptt_end_led_suppress() - return - except Exception: - pass - - # 会话未连接：先建链并发 Start，等待 started 后再自动开麦 - try: - if getattr(self, "AI_client_VC_isconnect", 0) != 1: - self._ptt_force_record_on_started = 1 - self.AI_VC_start() - return - except Exception: - self._ptt_force_record_on_started = 1 - try: - self.AI_VC_start() - except Exception: - pass - return - - # 会话还没 initialized：先发 Start，请求服务端进入 started - if getattr(self, "_vc_session_started", 0) != 1: - self._ptt_force_record_on_started = 1 - try: - try: - if t0: - print("[PTT] wait server.started (ms={})".format(utime.ticks_ms() - t0)) - except Exception: - pass - self.AI_client_VC_start_requset() - except Exception: - pass - return - - # 服务端仍在思考/TTS：先 cancel，在 btn 队列收到 canceled 后再开录（方案甲串行） - if self._vc_ptt_need_interrupt_wait(): - self._vc_begin_ptt_cancel_wait("ptt_begin") - return - - self._ptt_arm_uplink(t0) - - def _vc_effective_asr_text(self): - try: - t = (getattr(self, "AI_ASR_str", "") or "").strip() - if t: - return t - return (getattr(self, "_last_asr_result_text", "") or "").strip() - except Exception: - return "" - - def _vc_long_dialogue_auto_uplink_ok(self, reason): - """长对话 canceled/completed 自动开麦：须有有效 ASR，且防抖/勿在 LLM 无 TTS 后抢开。""" - try: - if reason not in ("canceled", "completed", "local.play_end", "audio.played"): - return True - if getattr(self, "Close_flag", 0) == 1: - return False - if reason == "canceled" and getattr(self, "_vc_pending_interrupt_uplink", 0) == 1: - pass - elif not self._vc_effective_asr_text(): - try: - print("[VC][uplink_ctrl] Deny START by {} cause=empty_asr".format(reason)) - except Exception: - pass - return False - if reason == "completed": - if getattr(self, "_turn_has_tts_started", 0) == 0: - try: - if getattr(self, "_llm_completed_ms", 0) or getattr(self.fsm, "turn", None) == VCTurnState.Thinking: - print("[VC][uplink_ctrl] Deny START by completed cause=llm_without_tts") - return False - except Exception: - pass - try: - now = utime.ticks_ms() - last = getattr(self, "_vc_last_auto_uplink_ms", 0) - if last and utime.ticks_diff(now, last) < 1200: - print("[VC][uplink_ctrl] Deny START by {} cause=auto_uplink_debounce".format(reason)) - return False - self._vc_last_auto_uplink_ms = now - except Exception: - pass - except Exception: - pass - return True - - def _vc_led_state_debug_enabled(self): - try: - if "vc_config" in globals() and vc_config: - return vc_config.get_bool("LED_STATE_DEBUG", True) - except Exception: - pass - return True - - def _vc_led_should_standby_idle(self): - """未录音且非 TTS 播中：待机/提示音/模式切换/单对话轮间应灭灯（DES-L3 idle），勿被 FSM.turn=Listening 误导。""" - try: - if getattr(self, "VC_record_state_flag", 0) != 0: - return False - if self._vc_tts_playback_active(): - return False - if getattr(self, "_ptt_await_server", 0) == 1: - return False - t = getattr(self.fsm, "turn", VCTurnState.Idle) - if t in ( - VCTurnState.Thinking, - VCTurnState.TtsReceiving, - VCTurnState.Playing, - VCTurnState.AwaitPlayedAck, - ): - return False - if getattr(self, "Close_flag", 0) == 1: - return True - if getattr(self, "VC_mode", -1) == 1: - return True - k = getattr(self, "_last_tts_kind", "") or "" - if k in ("prompt", "boot_prompt", "mode_switch_long", "mode_switch_ptt"): - return True - except Exception: - pass - return False - - def _vc_publish_led(self, ai_state, reason=""): - try: - if getattr(self, "_ptt_led_lock_listening", 0) == 1: - if ai_state != "listening": - if self._vc_led_state_debug_enabled(): - print("[LED] publish blocked ai={} reason={} (ptt_arm_lock)".format( - ai_state, reason)) - return - except Exception: - pass - try: - if self._vc_led_state_debug_enabled(): - print("[LED] publish ai={} reason={} VC_mode={} rec={} turn={}".format( - ai_state, reason, getattr(self, "VC_mode", -1), - getattr(self, "VC_record_state_flag", -1), - getattr(getattr(self, "fsm", None), "turn", "?"))) - except Exception: - pass - try: - EventMesh.publish("set_led_state", {"ai_state": ai_state}) - except Exception: - pass - - def _vc_led_sync_from_fsm(self): - """按 FSM/真实 TTS 播放恢复 AI 灯；本地 aud.play 短音不参与。""" - try: - if getattr(self, "_ptt_led_lock_listening", 0) == 1: - if getattr(self, "VC_record_state_flag", 0) == 0: - return - except Exception: - pass - try: - if self._vc_led_should_standby_idle(): - self._vc_publish_led("idle", "fsm_sync/standby_idle") - return - except Exception: - pass - try: - if getattr(self, "VC_mode", -1) == 1 and getattr(self, "_ptt_await_server", 0) == 1: - self._vc_publish_led("thinking", "fsm_sync/ptt_await") - return - except Exception: - pass - try: - if self._vc_tts_playback_active(): - self._vc_publish_led("playing", "fsm_sync/tts") - return - except Exception: - pass - try: - t = getattr(self.fsm, "turn", VCTurnState.Idle) - if t in (VCTurnState.Thinking, VCTurnState.TtsReceiving): - self._vc_publish_led("thinking", "fsm_sync/thinking") - elif t in (VCTurnState.Playing, VCTurnState.AwaitPlayedAck): - # 服务端已标记播放态但本地 PCM 尚未起播：仍显示思考，勿误绿灯 - self._vc_publish_led("thinking", "fsm_sync/await_tts") - elif t in (VCTurnState.Listening, VCTurnState.SpeechEnded, VCTurnState.SpeechStreaming): - self._vc_publish_led("listening", "fsm_sync/listening") - else: - self._vc_publish_led("idle", "fsm_sync/idle") - except Exception: - try: - self._vc_publish_led("idle", "fsm_sync/fallback") - except Exception: - pass - - def _uplink_request_start(self, reason="", *args, **kwargs): - # 统一入口：避免多处同时 start 造成冲突（尤其是 TTS/播放阶段误开麦导致回声触发与状态紊乱） - try: - if self.Close_flag == 1: - try: - if reason in ("canceled", "local.play_end", "audio.played", "key.press/continue"): - print("[VC][uplink_ctrl] Deny START by {} cause=Close_flag".format(reason)) - except Exception: - pass - return - except Exception: - return - - # 连续对话才允许“自动开新一轮上行” - try: - if self.VC_mode != 0: - try: - if reason in ("canceled", "local.play_end", "audio.played"): - print("[VC][uplink_ctrl] Deny START by {} cause=VC_mode({})".format(reason, self.VC_mode)) - except Exception: - pass - return - except Exception: - return - - try: - if self.VC_mode == 0 and not self._vc_long_dialogue_auto_uplink_ok(reason): - return - except Exception: - pass - - try: - if not self._vc_capture_allowed(reason): - try: - print("[VC][uplink_ctrl] Deny START by {} cause=playback_busy".format(reason)) - except Exception: - pass - return - except Exception: - return - - # ===== 业务门控（收敛到 FSM turn/session）===== - try: - if getattr(self.fsm, "session", VCSessionState.Disconnected) != VCSessionState.Started: - try: - if reason in ("canceled", "local.play_end", "audio.played"): - print("[VC][uplink_ctrl] Deny START by {} cause=fsm.session".format(reason)) - except Exception: - pass - return - turn = getattr(self.fsm, "turn", VCTurnState.Idle) - # 仅 Listening/SpeechEnded 允许开麦进入下一轮；其它阶段一律拒绝（避免抢跑打乱链路） - if turn not in (VCTurnState.Listening, VCTurnState.SpeechEnded): - try: - if reason in ("canceled", "local.play_end", "audio.played"): - print("[VC][uplink_ctrl] Deny START by {} cause=fsm.turn({})".format(reason, turn)) - except Exception: - pass - return - except Exception: - return - - # 若当前不在录音上行，才允许开新一轮 - if getattr(self, "VC_record_state_flag", 0) != 0: - # canceled 打断后：可能是僵死 flag=2（started 未真开麦），先停录复位再开麦 - if reason == "canceled": - try: - self._vc_clear_stale_record_state("uplink_start/canceled") - except Exception: - pass - if getattr(self, "VC_record_state_flag", 0) != 0: - try: - if reason in ("canceled", "local.play_end", "audio.played"): - print("[VC][uplink_ctrl] Deny START by {} cause=VC_record_state_flag({})".format( - reason, getattr(self, "VC_record_state_flag", 0))) - except Exception: - pass - return - - # 新一轮开始录音：允许上行（上一轮可能已置 silence 闸门） - if getattr(self, "record_data_save_flag", 0) == 0: - self._server_vad_silence = 0 - - print("[VC][uplink_ctrl] Request START by {}".format(reason)) - try: - if reason == "canceled" and getattr(self, "_vc_pending_interrupt_uplink", 0) == 1: - self._vc_pending_interrupt_uplink = 0 - print("[VC][uplink_ctrl] interrupt_uplink armed -> start (canceled)") - except Exception: - pass - self.VC_Speech_Start_now() - - def _uplink_request_stop(self, reason="", *args, **kwargs): - # 统一入口：避免多处同时 stop 造成冲突 - if getattr(self, "record_data_save_flag", 0) == 0: - self._server_vad_silence = 1 - - if getattr(self, "VC_record_state_flag", 0) == 0: - try: - if getattr(self, "VC_mode", 0) != 1: - print("[VC][long_idle] _uplink_request_stop skip (record_state==0) reason={}".format(reason)) - except Exception: - pass - return - - try: - if getattr(self, "VC_mode", 0) != 1: - print("[VC][long_idle] step=uplink_stop_exec reason={} VC_record_state_flag={}".format( - reason, getattr(self, "VC_record_state_flag", -1))) - except Exception: - pass - print("[VC][uplink_ctrl] Request STOP by {}".format(reason)) - self.VC_Speech_End() - - def _uplink_mark_start(self, first_len=0): - try: - if self._uplink_active == 1: - return - self._uplink_active = 1 - self._uplink_bytes = 0 - self._uplink_start_ms = utime.ticks_ms() - print("[VC][uplink] START len={} {} VC_record_state_flag={}".format( - first_len, self._vc_fsm_log_snip(), self.VC_record_state_flag)) - except Exception: - pass - - def _uplink_mark_end(self, reason=""): - try: - if self._uplink_active != 1: - return - dur = utime.ticks_diff(utime.ticks_ms(), self._uplink_start_ms) if self._uplink_start_ms else -1 - print("[VC][uplink] END reason={} bytes={} dur_ms={} {} VC_record_state_flag={}".format( - reason, self._uplink_bytes, dur, self._vc_fsm_log_snip(), self.VC_record_state_flag)) - except Exception: - pass - self._uplink_active = 0 - self._uplink_bytes = 0 - self._uplink_start_ms = 0 - - # 最近一次 TTS 的语义来源（dialog/prompt/mode_switch_long/mode_switch_ptt/...） - self._last_tts_kind = "" - # 下一个“无 dialog_id 的提示音 TTS”在 tts.started 时采用该 kind - self._pending_tts_kind = "" - - # ===== 状态机动作接口（骨架）===== - # 说明：统一把“副作用动作”收敛到这些方法，便于后续把 recv/timer 的 if/flag 逻辑迁移到 VCStateMachine。 - def request_send_start(self): - EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_start_requset}) - - def request_send_audio_create(self): - EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_send_speech}) - - def request_send_audio_completed(self): - EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_end_speech}) - - def request_send_audio_played(self): - EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_audio_played}) - - def request_send_synthesis(self, text): - try: - self.AI_TTS_str = text - except Exception: - pass - EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_synthesis}) - - def request_uplink_start(self, reason=""): - EventMesh.publish("publish_AI_btnInfo", {"func": self._uplink_request_start, "data": reason}) - - def request_uplink_stop(self, reason=""): - EventMesh.publish("publish_AI_btnInfo", {"func": self._uplink_request_stop, "data": reason}) - - def _fsm_apply_actions(self, actions): - try: - for a in actions: - if not a: - continue - if a[0] == "uplink_stop": - self.request_uplink_stop(a[1] if len(a) > 1 else "") - elif a[0] == "uplink_start": - self.request_uplink_start(a[1] if len(a) > 1 else "") - elif a[0] == "play_start": - if getattr(self, "VC_mode", -1) == 1: - if self._ptt_should_skip_remote_playback(): - if self._vc_led_state_debug_enabled(): - print("[VC][serial] skip play_start (ptt_interrupt)") - continue - try: - self.fsm.on_local_play_start() - except Exception: - pass - EventMesh.publish("publish_AI_btnInfo", {"func": self.VC_Play_Start}) - elif a[0] == "send_audio_played": - self.request_send_audio_played() - except Exception: - pass - - # AI - - def _print_hw_diag(self): - # 仅诊断使用：打印当前分支能看到的麦克/音频相关配置和运行时状态。 - # JSWM2 老分支没有 device_profile/_PROFILE，这里宽容处理 - try: - profile = _PROFILE if isinstance(_PROFILE, dict) else {} - except Exception: - profile = {} - dev_type_profile = profile.get("dev_type", None) - flags = profile.get("flags", {}) - hardware = profile.get("hardware", {}) - product_cfg = profile.get("product_cfg", {}) - vc_cfg = {} - try: - vc_cfg = product_cfg.get("vc", {}) - except Exception: - vc_cfg = {} - - print("[hw-diag] ===== AI_VC hardware diag =====") - print("[hw-diag] profile.dev_type =", dev_type_profile) - print("[hw-diag] profile.flags =", flags) - print("[hw-diag] profile.hardware =", hardware) - print("[hw-diag] product_cfg.vc =", vc_cfg) - - try: - print("[hw-diag] VC_record_samplerate =", self.VC_record_samplerate) - print("[hw-diag] VC_play_samplerate =", self.VC_play_samplerate) - except Exception: - pass - try: - print("[hw-diag] vad_silence_runtime =", self.vad_silence) - except Exception: - pass - try: - print("[hw-diag] record_data_save_flag =", self.record_data_save_flag) - print("[hw-diag] record_max_bytes =", self.record_max_bytes) - except Exception: - pass - - try: - g = self.record.gain_get() - except Exception as e: - g = "gain_get_error: {}".format(e) - print("[hw-diag] record.gain_get =", g) - try: - print("[hw-diag] type(record) =", type(self.record)) - print("[hw-diag] type(aud) =", type(self.aud)) - except Exception: - pass - print("[hw-diag] ===== end of hw-diag =====") - - def __init__(self): - # import voiceCall - # voiceCall.callStart("10086") - # return - self.http_get_config_flag = 0 - if self.http_get_config_flag == 1: - try: - self.http_get_config()#获取/更新配置参数 - except Exception as e: - usys.print_exception(e) - self.VC_30S_timer = osTimer() - self.ai_key_press_audio = osTimer() - self.VC_30S_timer.start(30000, 1, self.VC_30S_timer_cb) - self.online_kws = 0#是否开启线上关键词 - self.get_config_timer_count = 0 - self.record_data_save_flag = 0 #是否保存录音文件 - self.play_data_save_flag = 0#是否保存tts数据到文件 - self.record_fd = None - self.play_fd = None - self.is_VC_ACK = 0 #是否开启ack应答模式 - self.is_long_record = 0 #是否开启长录音模式（语音打断模式） - self.ai_OTA = OTA() - self.is_auth = 1 #是否开启dmp鉴权 - #self.VC_mode = 1 #0：短按连续问答模式 1：长按录音模式 - self.VC_mode = EventMesh.publish("get_storage_config","VC_mode")# - if self.VC_mode == None : - self.VC_mode = 0 - self.is_up_error= 0 #是否上传错误id - self.Close_flag =0 #=1时，连续对话播放完音频就退出 - # 服务端 VAD 结束标志：收到 vad.silence 后置 1，用于硬闸门禁止继续上行音频 - # （避免仍持续上行导致服务端推迟/抑制后续 TTS，从而一直停留在思考态蓝绿闪） - self._server_vad_silence = 0 - # 单轮状态标志：用于判定 completed 是否应自动进入下一轮 - self._turn_has_asr_completed = 0 - self._turn_has_vad_silence = 0 - self._turn_has_tts_started = 0 - # VC 会话是否已进入 started（服务端 session initialized） - # 重要：在收到 started 前发送音频帧会触发服务端 "session not initialized" → failed → 端侧重连死循环 - self._vc_session_started = 0 - # 开机/提示音：本轮 started 不自动开麦（避免问候 TTS 前误进 listening） - self._skip_started_uplink_once = 0 - self._last_tts_kind = "" - self._pending_tts_kind = "" - # Push-To-Talk（VAD关闭/单对话）是否已发送 audio.create - # 重要：协议要求 disable_vad=true 时必须 audio.create→音频帧→audio.completed - self._ptt_audio_created = 0 - # PTT：松手等服务器期间保持 thinking 勿回蓝闪聆听 - self._ptt_await_server = 0 - # 方案甲 btnInfo 串行：打断时先 cancel，收到 canceled（或超时）后再 audio.create - self._vc_serial_canceling = 0 - self._ptt_pending_arm = 0 - self._vc_cancel_deadline_ms = 0 - # PTT 打断：轮次号 + 开录前丢弃旧 TTS / 灯只显示聆听 - self._ptt_arm_epoch = 0 - self._ptt_interrupt_active = 0 - self._ptt_led_lock_listening = 0 - # 端侧主动发 synthesis 后到收到 tts.started 之间的窗口标志，用于阻止误开下一轮录音上行 - self._tts_pending = 0 - self._tts_pending_ms = 0 - # 各类兜底的节流时间戳（避免“兜底过激/连续触发”） - self._fb_last_thinking_restart_ms = 0 - self._fb_last_tts_pending_timeout_ms = 0 - # thinking 兜底：部分情况下服务端只返回 llm.completed 而不触发 tts.started/completed， - # 会导致一直停留在 thinking。此时超时自动回到聆听。 - self._llm_completed_ms = 0 - # 语音上行统计（真实发送到 VC 发送队列的开始/结束） - self._uplink_active = 0 - self._uplink_bytes = 0 - self._uplink_start_ms = 0 - - # 状态机骨架（后续逐步把旧逻辑迁移进来） - self.fsm = VCStateMachine(self) - - self.play_new_voice_flag =0 #播放文字留言flag - - self.wait_first_flag = 0#是否进入未读判定（只播报一次） - self.wait_first_timer = osTimer()#如果播放未读留言的时候未读则不要播放了 - self.wait_first_cnt = 0 - - self.wait_sort = 0#未读留言正在播第几条 - EventMesh.subscribe("play_tts",self.play_tts) - EventMesh.subscribe("play_new_voice",self.play_new_voice) - EventMesh.subscribe("play_his_voice",self.play_his_voice) - EventMesh.subscribe("play_wait_voice",self.play_wait_voice) - EventMesh.subscribe("play_single_voice",self.play_single_voice)#不带tts的播放（闹钟） - EventMesh.subscribe("play_music",self.play_music)#播放音乐 - EventMesh.subscribe("play_trans_tts",self.play_trans_tts)#接收翻译tts - EventMesh.subscribe("set_vad_silence",self.set_vad_silence) - EventMesh.subscribe("set_wait_first_flag",self.set_wait_first_flag) - EventMesh.subscribe("kws_even",self.kws_even) - EventMesh.subscribe("change_volume",self.change_volume) - EventMesh.subscribe("find_cmd",self.find_cmd) #查询命令 - EventMesh.subscribe("NLP_set_vad_silence",self.NLP_set_vad_silence) #设置聆听间隔 - EventMesh.subscribe("change_voice",self.change_voice) #改变音色 - EventMesh.subscribe("Close_VC",self.Close_VC) #外部结束对话 - EventMesh.subscribe("get_recv_voice",self.get_recv_voice) #获取留言播放状态 - EventMesh.subscribe("set_recv_voice",self.set_recv_voice) #设置留言播放状态 - self.voice_timer = osTimer()#语音留言10S按下按键确定是否播放的判断定时器 - self.voice_cnt = 0 #留言定时器计数 - self.voicerupt_flag = 0#留言选择按键接听标志位 - self.wait_voicerupt_flag = 0#未读留言选择按键接听标志位 - self.recv_voice = 0 #是否收到留言 - self.voice_notice = 0 #收到留言并播放则置为1，接下来要播放提示词 - self.voice_choose_flag = 0#当tts和语音留言冲突时，为了防止语音地址被误清，做标志位在close做判断 - - self.revoice_timer = osTimer()#语音留言10S按下按键确定是否回复的判断定时器 - self.revoice_cnt = 0 #回复留言定时器计数 - self.revoicerupt_flag = 0#留言选择按键接听标志位 - - self.music_timer = osTimer()#音乐播放定时器，等待tts播完才放音乐 - self.music_cnt = 0 #留言定时器计数 - - self.his_timer = osTimer()#历史留言播放定时器，等待tts播完再播留言 - self.his_cnt = 0#历史留言定时器技术 - - self.wait_timer = osTimer()#未读留言播放定时器，等待tts播完再播留言 - self.wait_cnt = 0#未读留言定时器技术 - - self.error_data = [] - self.asr_taskid = "" - self.VC_taskid = "" - self.VC_server = 2 #0柒牌正式服务器，1：测试服务器1,2：测试服务器2 - self.voice_flag = 0 - - #会议记录标志 - self.meet_flag = 0 - #AI翻译标志 - self.trans_flag = 0 - - #历史留言查看判断标志 - self.history_voice_flag = 0#公共变量判断 - #60S等待留言判断标志 - self.wait_voice_flag = 0#公共变量判断 - EventMesh.publish("AI_alarm_open")# - self.server_ip = EventMesh.publish("get_storage_config","server_ip")# - if self.server_ip == None: - if self.VC_server == 1:# - self.server_ip="118.145.158.207:22083" - elif self.VC_server == 2:# - self.server_ip="118.145.158.207:22082" - elif self.VC_server == 4:# - self.server_ip="118.145.158.207:22084" - elif self.VC_server == 5:# - self.server_ip="amp-wss.raisound.com" - elif self.VC_server == 6:# - self.server_ip="118.145.158.207:22080" - elif self.VC_server == 7:# - self.server_ip="118.145.158.207:22080" - elif self.VC_server == 8:# - self.server_ip="115.190.94.208:22080" - elif self.VC_server == 86:# - self.server_ip="118.145.158.207:22086" - else: - #self.server_ip="sevenpin.qipai.com.cn" - self.server_ip="118.145.158.207:22082" - # 默认走 constants.AMP_SERVER（由 default_srv.py 统一管理，可在 usr 覆盖） - try: - if constants.AMP_SERVER: - self.server_ip = constants.AMP_SERVER - except Exception: - pass - - #开机提示音 - self.power_on_tone = EventMesh.publish("get_storage_config","power_on_tone")# - try: - if "vc_config" in globals() and vc_config: - if not vc_config.get_bool("POWER_ON_TONE_ENABLED", True): - self.power_on_tone = None - except Exception: - pass - if self.power_on_tone == None: - try: - if "vc_config" in globals() and vc_config: - self.power_on_tone = vc_config.get_attr("POWER_ON_TONE_DEFAULT", None) - except Exception: - pass - - self.VC_record_TYPE = VC_record_TYPE_PCM #0:PCM 1:AMR - self.VC_play_TYPE = VC_play_TYPE_PCM #0:PCM 1:mp3 - self.VC_record_samplerate = 16000 - self.VC_play_samplerate = 16000 - self.VAD_data_len = 640 #10ms数据 - self.is_VAD = 0#=1打开vad打断 - # ARG_sample_rate, // 采样率 - # ARG_channel, // 通道数 - # ARG_frame_size, // 每帧字节数 - # ARG_energy_threshold, // 能量阈值 - # ARG_voice_time, // 语音持续时间阈值(ms) - # ARG_silence_time, // 静音持续时间阈值(ms) - # ARG_silence_start_threshold, // 静音开始阈值(ms) - self.vad = None - if self.is_VAD == 1: - self.vad = VAD.VAD(self.VC_record_samplerate ,1,self.VAD_data_len,600,100,300,100) - self.speech_rate = EventMesh.publish("get_storage_config","speech_rate")#语速配置 - if self.speech_rate == None: - self.speech_rate = 0 - - self.vad_silence = EventMesh.publish("get_storage_config","vad_silence")#语速配置 - if self.vad_silence == None: - try: - if "vc_config" in globals() and vc_config: - self.vad_silence = vc_config.get_int("VAD_SILENCE_DEFAULT", 300) - else: - self.vad_silence = 300 - except Exception: - self.vad_silence = 300 - - self.VC_TTS_voice_type = EventMesh.publish("get_storage_config","VC_TTS_voice_type")#音色配置 - if self.VC_TTS_voice_type == None : - try: - if "vc_config" in globals() and vc_config: - cfg = vc_config.get_attr("VC_TTS_VOICE_TYPE_DEFAULT", None) - if isinstance(cfg, dict): - self.VC_TTS_voice_type = cfg - except Exception: - pass - # 若配置未提供，沿用原逻辑内置兜底（保持兼容） - if self.VC_TTS_voice_type == None: - self.VC_TTS_voice_type = { - "女":{"bot_name":"柒妹妹","voice":"zh_female_wanwanxiaohe_moon_bigtts","lang":"zh-cn"}, - "男":{"bot_name":"柒先生","voice":"zh_male_wennuanahu_moon_bigtts","lang":"zh-cn"}, - "ç«¥":{"bot_name":"柒乐乐","voice":"ICL_zh_female_huoponvhai_tob","lang":"zh-cn"}, - } - - # self.VC_TTS_voice_type = {0:"zh_female_xinlingjitang_moon_bigtts",1:"zh_male_wennuanahu_moon_bigtts",2:"ICL_zh_female_huoponvhai_tob", - # 3:"aliyun::longjiaxin",4:"zh_female_daimengchuanmei_moon_bigtts"} - #self.VC_TTS_voice_type = ["aliyun::longxiaoxia","aliyun::longnan","aliyun::longling"] - - #EventMesh.publish("set_storage_config",["VC_TTS_voice_type",self.VC_TTS_voice_type]) - - try: - self.VC_TTS_voice_key = EventMesh.publish("get_storage_config","VC_TTS_voice_key")#音色配置 - if self.VC_TTS_voice_key == None : - try: - if "vc_config" in globals() and vc_config: - self.VC_TTS_voice_key = vc_config.get_attr("VC_TTS_VOICE_KEY_DEFAULT", None) - except Exception: - pass - if not self.VC_TTS_voice_key: - self.VC_TTS_voice_key = "男" - if self.VC_TTS_voice_key in self.VC_TTS_voice_type: - pass - else: - self.VC_TTS_voice_key = list(self.VC_TTS_voice_type.keys())[0] - print("音色配置：{}:{}\r\n\r\n".format(self.VC_TTS_voice_key,self.VC_TTS_voice_type[self.VC_TTS_voice_key])) - self.VC_TTS_volume = EventMesh.publish("get_storage_config","VC_TTS_volume")#音量配置 - except Exception as e: - usys.print_exception(e) - #配置词 - self.commands_mode = 2 #=1用大模型判断命令，=0用服务器本地插件判断命令 =2用设备本地判断方式 - self.commands = [] - if self.VC_TTS_volume == None: - self.VC_TTS_volume = 100 - #可ota配置的参数变量 - self.emotion = EventMesh.publish("get_storage_config","emotion")#情绪 - if self.emotion == None: - self.emotion = "" - self.ota_update_notice = EventMesh.publish("get_storage_config","ota_update_notice")#OTA升级提示语 - if self.ota_update_notice == None: - self.ota_update_notice = "" - self.AI_audio_queue_thread_id = 0 - self.AI_audio_queue_thread_lock = 0 - self.AI_VC_send_data_queue_thread_id = 0 - self.AI_VC_send_data_queue_thread_lock = 0 - self.AI_ASR_send_data_queue_thread_id = 0 - self.AI_ASR_send_data_queue_thread_lock = 0 - self.AI_record_tmp_queue = Queue(100) - self.AI_ASR_send_data_queue = Queue(200) - self.AI_VC_send_data_queue = Queue(100) - self.AI_audio_queue = Queue(100) - #self.AI_WS_connect_Queue = Queue(100) #释放用 - self.AI_TTS_str = "" - self.AI_ASR_str = "" - self.AI_ASR_flag = 0 - self.AI_ASR_str_tmp = "" - self.AI_ASR_send_buf_len = 0 - self.AI_ASR_text_tmp = "" - self.AI_ASR_text_repet_count = 0 - # #开始线程 - # if self.AI_audio_queue_thread_id ==0: - # self.AI_audio_queue_thread_id = _thread.start_new_thread(self.AI_audio_queue_thread, ()) - # else: - # print("AI_VC_start :AI_audio_queue_thread_id != 0 ") - # if self.AI_VC_send_data_queue_thread_id == 0: - # self.AI_VC_send_data_queue_thread_id = _thread.start_new_thread(self.AI_VC_send_data_queue_thread, ()) - - self.VC_dialog_id = ""# - self.VC_message_id=None - self.VC_task_id =None - self.AI_client_VC = None - self.AI_client_VC_recv_id = 0 - self.AI_client_VC_recv_lock = 0 - self.AI_client_VC_isconnect =0 - self.AI_client_VC_debug =0 - - self.asr_state = 0 # 预留：ASR 侧状态（当前未参与分支） - self.key_state = 0 - - self.AI_client_asr = None - self.AI_client_asr_recv_id = 0 - self.AI_client_asr_recv_lock = 0 - self.AI_client_asr_isconnect =0 - self.AI_ASR_timer = osTimer() - self.AI_ASR_timer_count = 0 - self.AI_client_asr_debug =0 - self.AI_ASR_check_timeout = 0 - self.AI_ASR_error_timeout = 0 - self.VC_Thinking_timerout = 0 - - - self.record_pcm = None - self.play_pcm = None - self.VC_record_thread_id = 0 - self.VC_record_thread_lock = 0 - self.VC_pingping_timerout = 0 - self.asr_pingping_timerout = 0 - self.factory_timer = osTimer() - self._1S_timer = osTimer() - self.Audio_buf_size = 20*1024 - self.record = audio.Record() - #self.record.end_callback(self.record_callback) - self.record.gain_set(4,10) - #self.record.gain_set(4,12) - self.aud = audio.Audio(0) - self.aud.setCallback(self.audio_cb) - # 确保 record/aud 已创建后再打印硬件/音频诊断信息（需要 gain_get/type(record)） - try: - self._print_hw_diag() - except Exception as e: - usys.print_exception(e) - #self.aud.setVolume(self.VC_TTS_volume //10) - self.audio_play_state = 1 #音频播放完状态 - self.close_timerout = 0 - self._idle_deep_sec = 0 - self._vc_idle_deep_armed_suppress = 0 - self.LocalRespondingEnded_timerout = 0 - - self.VC_Play_flag = 0 - self.play_flag = 0 - self.VC_record_state_flag = 0 - self.AI_key_long_press_flag = 0 - self._asr_completed_tone_done = 0 # asr.completed 本地 think 提示每轮仅一次 - self._last_asr_result_text = "" # 保留最近 asr.result，防 VC_Speech_End 早于 asr.completed 清空 - self._vc_last_auto_uplink_ms = 0 # 长对话自动开麦防抖 - self._vc_local_play_active = 0 # aud.play 本地短音占用（与采集互斥） - self._vc_local_play_until_ms = 0 - self._vc_pending_interrupt_uplink = 0 # 长对话短按打断：等 canceled 再开麦 - self.websocket_error_count = 0 - self.AI_new_voice_url = "" #未读语音地址/音乐播放地址/闹钟播放标志 - self.music_play_flag = 0 #音乐播放标志 - self.alarm_play_flag = 0 #闹钟播放标志 - self.tts_in_data_len = 0 - self.tts_out_data_len = 0 - self._tts_first_write_ms = 0 - self._tts_play_check_log_ms = 0 - self._need_audio_played_ack = 0#tts.completed 后需在本地播报结束后上报 audio.played；勿用 tts_in/out：tts.completed 会再调 VC_Play_Start 把二者清零 - self.VC_satrt_flag = 0 - self.VC_stop_play_audio_flag = 0 - self.VC_satrt_kws_flag = 0 - self.session_id = "" - self.stop_voice_timer=osTimer() - self.pwk_cnt_voice = 0 #不听语音后，积攒的次数，需要一起播放 - self.pwk_notice_flag = 0#是否处于积攒留言状态(用来实现单次提醒) - self.is_noread_voice = EventMesh.publish("get_storage_config","is_noread_voice") - if self.is_noread_voice == None: - self.is_noread_voice = 0#是否还有未读留言（开机判断） - self.is_ruppt_tts = 0#是否已经打断tts播放 - EventMesh.subscribe("AI_key_press", self.AI_key_press) - EventMesh.subscribe("BT_AI_key_press", self.BT_AI_key_press) - EventMesh.subscribe("stop_AI_record", self.stop_AI_record) - EventMesh.subscribe("AI_key_release", self.AI_key_release) - EventMesh.subscribe("pwk_key_double_press", self.pwk_key_double_press) - EventMesh.subscribe("AI_key_long_press", self.AI_key_long_press) - EventMesh.subscribe("pwk_key_release", self.pwk_key_release) - EventMesh.subscribe("get_AI_audio_queue", self.get_AI_audio_queue) - EventMesh.subscribe("get_VC_satrt_flag", self.get_VC_satrt_flag) - EventMesh.subscribe("get_vc_stream_active", self.get_vc_stream_active) - EventMesh.subscribe("get_vc_tts_playback_active", self.get_vc_tts_playback_active) - EventMesh.subscribe("battery_priority_mute_vc", self.battery_priority_mute_vc) - EventMesh.subscribe("VC_error", self.VC_error) - EventMesh.subscribe("audio_test", self.audio_test) - EventMesh.subscribe("audio_test_close", self.audio_test_close) - EventMesh.subscribe("set_record_data_save_flag", self.set_record_data_save_flag) - EventMesh.subscribe("set_VC_TTS_voice_type", self.set_VC_TTS_voice_type) - EventMesh.subscribe("SET_TTS_VOLUME", self.set_tts_volume) - EventMesh.subscribe("set_VC_stop_play_audio_flag", self.set_VC_stop_play_audio_flag) - EventMesh.subscribe("set_yuyinaudio_flag",self.set_yuyinaudio_flag) - EventMesh.subscribe("get_yuyinaudio_flag",self.get_yuyinaudio_flag) - EventMesh.subscribe("set_pwk_cnt_voice",self.set_pwk_cnt_voice) - EventMesh.subscribe("get_pwk_cnt_voice",self.get_pwk_cnt_voice) - EventMesh.subscribe("exit_voice_message_mode",self.exit_voice_message_mode) #超时未退出语音留言模式 - EventMesh.subscribe("come_meet_mode",self.come_meet_mode)#进入会议模式 - EventMesh.subscribe("exit_meet_mode",self.exit_meet_mode)#退出会议模式 - EventMesh.subscribe("come_trans_mode",self.come_trans_mode)#进入翻译模式 - EventMesh.subscribe("exit_trans_mode",self.exit_trans_mode)#退出翻译模式 - EventMesh.subscribe("set_trans_flag",self.set_trans_flag)#设置翻译状态 - EventMesh.subscribe("get_audio_play_state",self.get_audio_play_state) - EventMesh.subscribe("set_voicerupt_flag",self.set_voicerupt_flag)#按键中断触发留言播放 - EventMesh.subscribe("set_wait_voicerupt_flag",self.set_wait_voicerupt_flag)#按键中断触发未读留言播放 - EventMesh.subscribe("get_recv_voice",self.get_recv_voice)#获取是否收到留言 - EventMesh.subscribe("rate_change",self.rate_change)#直接修改语速 - EventMesh.subscribe("get_history_voice_flag",self.get_history_voice_flag)#获取查找历史留言状态 - EventMesh.subscribe("set_history_voice_flag",self.set_history_voice_flag)#修改查找历史留言状态 - EventMesh.subscribe("clean_unread_vocie",self.clean_unread_vocie)#修改查找历史留言状态 - EventMesh.subscribe("get_wait_voice_flag",self.get_wait_voice_flag)#获取60S等待留言播放定时器状态 - EventMesh.subscribe("set_wait_voice_flag",self.set_wait_voice_flag)#修改60S等待留言播放定时器状态 - EventMesh.subscribe("rate_move",self.rate_move)#NLP修改语速 - EventMesh.subscribe("find_rate",self.find_rate)#NLP查询语速 - EventMesh.subscribe("set_pwk_notice_flag",self.set_pwk_notice_flag)#设置未读状态 - # EventMesh.subscribe("get_voicerupt_flag",self.get_voicerupt_flag)#获取留言播放状态 - EventMesh.subscribe("chenggong_tone",self.chenggong_tone) - - pass - EventMesh.publish("set_led_state", {"ai_state": "idle"}) - self.dialog_attributes_provider = EventMesh.publish("get_storage_config","dialog_attributes_provider")#智能体类型配置 - if self.dialog_attributes_provider == None: - self.dialog_attributes_provider = "coze" - self.outbound_attributess_provider = EventMesh.publish("get_storage_config","outbound_attributess_provider")#tts服务选择 - if self.outbound_attributess_provider == None: - self.outbound_attributess_provider = "volcengine" - self.inbound_attributess_provider = EventMesh.publish("get_storage_config","inbound_attributess_provider")#tts服务选择 - if self.inbound_attributess_provider == None: - self.inbound_attributess_provider = "volcengine" - - self.bot_name = "柒先生" - try: - if "bot_name" in self.VC_TTS_voice_type[self.VC_TTS_voice_key]: - self.bot_name = self.VC_TTS_voice_type[self.VC_TTS_voice_key]["bot_name"] - else: - print("not bot_name!!!!!!!!!!!!!!!!!!!!!!") - except Exception as e: - usys.print_exception(e) - try: - bt_ota_flag=0 - try: - if "vc_config" in globals() and vc_config and vc_config.get_bool("BT_OTA_ENABLED", False): - bt_ota_flag = self.ai_OTA.bt_ota()#先检查蓝牙是否要升级 - except Exception: - pass - - cat1_ota_flag = self.ai_OTA.min_sys_ota() - if bt_ota_flag==1 or cat1_ota_flag == 1: - #if 0: - if self.ota_update_notice != "": - self.AI_TTS_str = self.ota_update_notice - else: - self.AI_TTS_str = "检测到有新版本需要升级啦！升级时可千万别断电或乱按哦，升级完成后设备会自动重启，重启就代表升级成功" - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - self.Close_flag = 1 - # if self.ai_OTA.min_sys_ota()==2: - # self.AI_TTS_str = "检测到有新版本需要升级啦！电量低于40%，无法保证升级正常执行，请充满电后重启设备再次升级！" - # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - else: - try: - self.ai_OTA.ota_result() - self.ai_OTA.pots_ICCID() - except Exception as e: - usys.print_exception(e) - #开机提示语，如果联网失败，直接提示信号弱 - - - #EventMesh.publish("get_storage_config",["powerRestart",1]) - powerRestart = EventMesh.publish("get_storage_config","powerRestart")#=1非正常重启 - if powerRestart == None: - powerRestart = 0 - if powerRestart == 1: - EventMesh.publish("set_storage_config",["powerRestart",0]) - - else: - pass - self.AI_TTS_str = self.power_on_tone.replace("bot_name", self.bot_name) - print("开机提示：{}".format(self.AI_TTS_str)) - try: - self._pending_tts_kind = "boot_prompt" - self._last_tts_kind = "boot_prompt" - self._skip_started_uplink_once = 1 - except Exception: - pass - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - - except Exception as e: - usys.print_exception(e) - def set_VC_stop_play_audio_flag(self,event = None, msg = None): - self.VC_stop_play_audio_flag = msg - print("self.VC_stop_play_audio_flag:{}".format(self.VC_stop_play_audio_flag)) - - def rate_change(self,event,msg):#直接设置语速（数字） - self.speech_rate = msg - EventMesh.publish("set_storage_config",["speech_rate",self.speech_rate]) - print("set_speech_rate:{}".format(self.speech_rate)) - def rate_move(self,event,msg):#设置语速(模糊(å¿«/慢一点)) - if msg == 1 and self.speech_rate<50:#增大 - self.speech_rate += 10 - else:#减小 - if self.speech_rate >-50: - self.speech_rate -= 10 - EventMesh.publish("set_storage_config",["speech_rate",self.speech_rate]) - print("set_speech_rate:{}".format(self.speech_rate)) - def find_rate(self,event,msg):#查询APP语速 - print("查询语速处理：原始数据为：{}".format(self.speech_rate)) - y = 1 - if self.speech_rate >= -50 and self.speech_rate < 0:#负数 - y = 1 - (-0.01 * self.speech_rate) - elif self.speech_rate <= 50 and self.speech_rate >= 0:#正数 - y = 0.02 * self.speech_rate + 1.0 - print("处理后数据为{}".format(y)) - rate = str(y) + "倍" - return rate - def set_pwk_notice_flag(self,event,msg):#设置未读状态 - self.pwk_notice_flag = msg - #获取60S等待留言播放定时器状态 - def get_wait_voice_flag(self,event,msg): - return self.wait_voice_flag - def set_wait_voice_flag(self,event,msg): - # print("未读留言标记修改为:{}".format(msg)) - self.wait_voice_flag = msg - - #查看历史留言状态修改 - def get_history_voice_flag(self,event,msg): - return self.history_voice_flag - def set_history_voice_flag(self,event,msg): - # print("历史留言标记修改为:{}".format(msg)) - self.history_voice_flag = msg - def clean_unread_vocie(self,event,msg): - self.pwk_cnt_voice = 0 - self.pwk_notice_flag = 0 - EventMesh.publish("set_app_new_voice_flag",0) - def audio_test(self,event = None, msg = None): - self._uplink_request_start("audio_test") - self.VC_Play_Start() - def audio_test_close(self,event = None, msg = None): - self._uplink_request_stop("audio_test_close") - self.VC_Play_End() - if self.record_data_save_flag == 1: - if self.record_fd : - self.record_fd.close() - self.record_fd = None - def set_record_data_save_flag(self,event = None, msg = None): - self.record_data_save_flag = msg - def post_processor_after_instantiation(self): - pass - def get_AI_audio_queue(self, event, msg): - return self.AI_audio_queue - def get_VC_satrt_flag(self, event, msg): - return self.VC_satrt_flag - #检查有没有意外打开ai部分功能 - def check_vc_start(self): - print("check_vc_start:{} {} {} {}".format(self.VC_satrt_flag,self.VC_record_state_flag,self.AI_client_VC_isconnect,self.VC_Play_flag)) - if self.VC_satrt_flag == 0: - if self.VC_record_state_flag != 0 or self.AI_client_VC_isconnect == 1 or self.VC_Play_flag == 1: - self.Close() - def VC_30S_timer_cb(self, *args): # - EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_30S_timer_cb_}) - - def VC_30S_timer_cb_(self): - self.get_config_timer_count += 1 - if self.get_config_timer_count >= 2*60*2:#2小时更新配置参数 - self.get_config_timer_count = 0 - if self.http_get_config_flag == 1: - try: - self.http_get_config()#获取/更新配置参数 - except Exception as e: - usys.print_exception(e) - self.check_vc_start()#30秒检查一下有没有关闭 - #提交错误id - def ud_error(self,error_data): - response = None - if self.is_up_error == 1: - try: - # currentVersion = versions.Build_Time - # currentVersion = "7PIN-"+currentVersion[:10]+"-"+currentVersion[11:-3] - url = "http://{}/health?task_id={}".format(self.server_ip,error_data[0]+error_data[1]) - print(url) - response = request.get(url,timeout=5) - if response.status_code == 200: - - print("ud_error:ok") - else: - print("ud_error error!!!!") - for i in response.text: - print(i) - response.close() - except Exception as e: - usys.print_exception(e) - if response != None: - response.close() - #获取配置参数 - def http_get_config(self, event=None, msg=None): - print("http_get_config") - - response = None - #http://180.184.206.142/bz-api/dev/config?deviceId=xxx - try: - url = (constants.DMP_BASE + "/dev/config/{}").format(modem.getDevImei()) - - - print(url) - response = request.get(url,timeout=5) - if response.status_code == 200: - data = response.json() - print("http_get_config:{}".format(data)) - - if data: - try: - if data["code"] == 20000:#有配置 - if data["data"] : - if "version" in data["data"]: - config_version = EventMesh.publish("get_storage_config","config_version")#音量配置 - if config_version != data["data"]["version"]: - config_version = data["data"]["version"] - EventMesh.publish("set_storage_config",["config_version",config_version]) - if "VC_TTS_voice_type" in data["data"]: - self.VC_TTS_voice_type = data["data"]["VC_TTS_voice_type"] - EventMesh.publish("set_storage_config",["VC_TTS_voice_type",self.VC_TTS_voice_type]) - print("配置VC_TTS_voice_type成功") - if "server_ip" in data["data"]: - self.server_ip = data["data"]["server_ip"] - EventMesh.publish("set_storage_config",["server_ip",self.server_ip]) - print("配置server_ip成功") - if "power_on_tone" in data["data"]: - self.power_on_tone = data["data"]["power_on_tone"] - EventMesh.publish("set_storage_config",["power_on_tone",self.power_on_tone]) - print("配置power_on_tone成功") - if "dialog_attributes_provider" in data["data"]: - self.dialog_attributes_provider = data["data"]["dialog_attributes_provider"] - EventMesh.publish("set_storage_config",["dialog_attributes_provider",self.dialog_attributes_provider]) - if "outbound_attributess_provider" in data["data"]: - self.outbound_attributess_provider = data["data"]["outbound_attributess_provider"] - EventMesh.publish("set_storage_config",["outbound_attributess_provider",self.outbound_attributess_provider]) - print("配置outbound_attributess_provider成功") - if "inbound_attributess_provider" in data["data"]: - self.inbound_attributess_provider = data["data"]["inbound_attributess_provider"] - EventMesh.publish("set_storage_config",["inbound_attributess_provider",self.inbound_attributess_provider]) - print("配置inbound_attributess_provider成功") - if "speech_rate" in data["data"]: - self.speech_rate = EventMesh.publish("SET_SPEECH_RATE",float(data["data"]["speech_rate"])) - EventMesh.publish("set_storage_config",["speech_rate",self.speech_rate]) - print("配置speech_rate成功") - if "emotion" in data["data"]: - self.emotion = data["data"]["emotion"] - EventMesh.publish("set_storage_config",["emotion",self.emotion]) - print("配置emotion成功") - if "ota_update_notice" in data["data"]: - self.ota_update_notice =data["data"]["ota_update_notice"] - EventMesh.publish("set_storage_config",["ota_update_notice",self.ota_update_notice]) - print("配置ota_update_notice成功") - if "VC_TTS_volume" in data["data"]: - self.VC_TTS_volume =int(data["data"]["VC_TTS_volume"]) - EventMesh.publish("set_storage_config",["VC_TTS_volume",self.VC_TTS_volume]) - print("配置VC_TTS_volume成功") - - except Exception as e: - usys.print_exception(e) - else: - print("status_code:{}".format(response.status_code)) - print("http_get_config:error!!!!!!!!!!") - response.close() - except Exception as e: - usys.print_exception(e) - if response != None: - response.close() - - def audio_cb(self,event): - if event == 0: - EventMesh.publish(constants.EVENT.SYSTEM_UP)#唤醒休眠 - self.audio_play_state =1 - #EventMesh.publish("set_audio_state", 1) - print('audio-play start.') - # 本地 aud.play（开录/思考短音等）不改动 AI 灯，仅 TTS 流由 VC_Play_Start 控绿灯 - if self._vc_led_state_debug_enabled(): - print("[LED] audio_cb start skip (local_play)") - return - elif event == 7: - self.audio_play_state = 0 - #EventMesh.publish("set_audio_state", 0) - print('audio-play finish.!!!!!!') - try: - if getattr(self, "VC_mode", -1) == 1 and getattr(self, "_ptt_await_server", 0) == 1: - self._vc_publish_led("thinking", "audio_cb/finish/ptt_await") - return - except Exception: - pass - try: - if getattr(self, "VC_record_state_flag", 0) != 0: - self._vc_publish_led("listening", "audio_cb/finish/recording") - return - except Exception: - pass - try: - if self._vc_tts_playback_active(): - if self._vc_led_state_debug_enabled(): - print("[LED] audio_cb finish skip (tts_playback)") - return - except Exception: - pass - try: - if getattr(self, "_ptt_led_lock_listening", 0) == 1: - return - except Exception: - pass - self._vc_led_sync_from_fsm() - def set_tts_volume(self,event,msg): - EventMesh.publish("set_storage_config",["VC_TTS_volume",msg])#音量配置 - self.VC_TTS_volume = msg - print("set_tts_volume:{} {}".format(self.VC_TTS_volume,msg)) - if int(msg) <= 10: #最小一定要设置1要不然没有声音 - volume = 10 //10 - self.aud.setVolume(volume) - else: - self.aud.setVolume(self.VC_TTS_volume //10) - pass - def get_yuyinaudio_flag(self, event=None, msg = None): - return self.voice_flag - - def set_yuyinaudio_flag(self, event=None, msg = None): - self.voice_flag=0 - - #留言主动拒听 - def get_pwk_cnt_voice(self, event=None, msg = None): - return self.pwk_cnt_voice - - def set_pwk_cnt_voice(self, event=None, msg = None): - self.pwk_cnt_voice=msg - print("设置未读次数为：{}".format(self.pwk_cnt_voice)) - - #会议 - def get_meetaudio_flag(self, event=None, msg = None): - return self.meet_flag - def set_meetaudio_flag(self, event=None, msg = None): - self.meet_flag = 0 - - def get_audio_play_state(self, event=None, msg = None): - return self.audio_play_state - - - - def come_meet_mode(self, event=None, msg = None): - if self.meet_flag == 0: #还是为0就提示进入会议纪要模式 - EventMesh.publish("send_meet_open_flag_to_iot")#创建会议 - self.meet_flag=1 - exit_tts ="开启会议纪要，正在录音" - EventMesh.publish("play_tts",[exit_tts,1]) - utime.sleep_ms(4000) - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_key_press}) - pass - def exit_meet_mode(self, event=None, msg = None): - if self.meet_flag == 1: #还是为1就提示已退出会议纪要模式 - EventMesh.publish("send_meet_end_flag_to_iot")#结束会议 - self.meet_flag=0 - EventMesh.publish("stop_voice_record") #关闭语音录音 - exit_tts ="录音结束，已退出会议纪要模式" - EventMesh.publish("play_tts",[exit_tts,1]) - pass - def come_trans_mode(self, event=None, msg = None): - if self.trans_flag == 0: #还是为0就提示进入AI翻译模式 - EventMesh.publish("send_trans_come_flag_to_iot")#进入翻译 - pass - def exit_trans_mode(self, event=None, msg = None): - if self.trans_flag == 1: #还是为1就提示已退出AI翻译模式 - EventMesh.publish("stop_voice_record") #关闭语音录音 - exit_tts ="已退出AI翻译模式" - EventMesh.publish("play_tts",[exit_tts,1]) - self.trans_flag=0 - EventMesh.publish("send_trans_end_flag_to_iot")#退出翻译 - pass - def set_trans_flag(self,event,msg): - self.trans_flag = msg - def exit_voice_message_mode(self, event=None, msg = None): - if self.voice_flag == 1: #还是为1就提示已退出会议纪要模式 - self.voice_flag=0 - exit_tts ="已退出语音留言模式" - EventMesh.publish("play_tts",[exit_tts,1]) - pass - def ai_key_press_audio_cb1(self, event=None, msg = None): - print("播放按键音!!!!!!!!!!") - # EventMesh.publish("audio_montage_play",["chenggong.mp3"]) - # self.chenggong_tone() - self.aud.aud_tone_play(10,400)#播放按键音 - def ai_key_press_audio_cb2(self, event=None, msg = None): - EventMesh.publish("audio_montage_play",["chenggong.mp3"]) - # self.chenggong_tone() #播放按键音 - - def AI_key_press(self, event=None, msg = None):#按键按下 - # 按键时打印 LED 聚合状态/最终模式（用于定位“绿灯不灭/闪一下”） - try: - if "vc_config" in globals() and vc_config and vc_config.get_bool("KEY_LED_DEBUG", False): - try: - led = EventMesh.publish("get_led_debug_state") - except Exception: - led = None - try: - t = getattr(self.fsm, "turn", None) - except Exception: - t = None - print("[KEY] AI_key_press VC_mode={} {} turn={} led={}".format( - getattr(self, "VC_mode", -1), self._vc_fsm_log_snip(), t, led)) - except Exception: - pass - # 单对话 PTT / 长对话分支（按键短按提示音已关，避免与开录等提示音叠在一起） - try: - if self._ai_key_press_ptt_mode(): - return - except Exception: - pass - if self._ai_key_press_long_dialogue_interrupt(): - return - try: - if self._ai_key_press_long_dialogue_when_session_ready(): - return - except Exception: - pass - try: - if self._ai_key_press_long_dialogue_wake(): - EventMesh.publish(constants.EVENT.SYSTEM_UP) - return - except Exception: - pass - - EventMesh.publish(constants.EVENT.SYSTEM_UP)#唤醒休眠 - if EventMesh.publish("get_sos_flag")== 1: - return - if self.voice_flag == 1: #语音留言模式 - self.stop_voice_timer.stop() #关闭语音留言模式定时器 - # EventMesh.publish("stop_AI_record") - self.Close() - self.revoice_cnt = 0 - self.revoice_timer.stop()#回复定时器关闭，计数器清零 - EventMesh.publish("start_voice_record",2)#开始语音录音 - return - elif self.meet_flag == 1:#会议纪要模式 - self.Close() - EventMesh.publish("start_voice_record",3)#开始会议录音 - return - elif self.trans_flag == 1:#AI翻译模式 - return - - EventMesh.publish("stop_voice_record") #关闭语音录音 - print("AI_key_press: unhandled mode branch") - self.AI_key_long_press_flag = 0 - def BT_AI_key_press(self, event=None, msg = None):#按键按下 - EventMesh.publish(constants.EVENT.SYSTEM_UP)#唤醒休眠 - if EventMesh.publish("get_sos_flag")== 1: - return - if self.voice_flag == 1: #语音留言模式 - tts_str = "语音留言模式，长按按钮开始录音，松开按钮结束" - EventMesh.publish("play_tts",[tts_str,1]) - return - - EventMesh.publish("stop_voice_record") #开启静默需要关闭 - print("AI_key_press") - self.get_config_timer_count = 0 - self.key_state = 1 - # EventMesh.publish("audio_montage_play",["chenggong.mp3"]) - # self.chenggong_tone() #播放按键音 - # 长对话：蓝牙按键同单按 - # - 对话进行中：打断/续聊（复用 dialog_id） - # - 已停止/未 started：走新会话流程 - try: - if self.VC_mode == 0: - if getattr(self, "AI_client_VC_isconnect", 0) == 1 and getattr(self, "_vc_session_started", 0) == 1: - EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_RequestToSpeak}) - return - except Exception: - pass - - self.Close() - - if self.VC_mode == 1: #长按对话模式 接收到蓝牙耳机按键事件 - - tts_str = "请您按住圆形的按钮开始讲话，讲完松开即可 " - EventMesh.publish("play_tts",[tts_str,1]) - return - # if dev_type == "7PIN": - # self.chenggong_tone() - #self.aud.aud_tone_play(10,200)#播放按键音 - self.AI_TTS_str = ""#"你好，有什么吩咐" - self.AI_ASR_str = "" - self.AI_ASR_str_tmp = "" - # if self.VC_state != VC_State_None: - # EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) - # EventMesh.publish("publish_AI_btnInfo",{"func":utime.sleep_ms,'data':(10)}) - self._uplink_request_stop("BT_AI_key_press/compat2") - self._uplink_request_start("BT_AI_key_press/compat2") - - self.AI_VC_start() - - - # self.Close() - # self.AI_VC_start() - self.AI_key_long_press_flag = 0 - def stop_AI_record(self, event=None, msg=None):#停止AI对话 - if self.VC_satrt_flag == 1: - self.get_config_timer_count = 0 - self.AI_TTS_str = ""#"你好，有什么吩咐" - self.AI_ASR_str = "" - self.AI_ASR_str_tmp = "" - self.AI_key_long_press_flag = 0 - EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) - self.request_uplink_stop("stop_AI_record") - count = 0 - while True: - if count >= 20 or self.VC_satrt_flag == 0: - break - count+=1 - utime.sleep_ms(500) - - def exit_voice_message_timer(self,*argc): - EventMesh.publish("exit_voice_message_mode") - - def AI_key_release(self, event=None, msg=None):#按键弹起 - try: - if "vc_config" in globals() and vc_config and vc_config.get_bool("KEY_LED_DEBUG", False): - try: - led = EventMesh.publish("get_led_debug_state") - except Exception: - led = None - try: - t = getattr(self.fsm, "turn", None) - except Exception: - t = None - print("[KEY] AI_key_release VC_mode={} {} turn={} led={}".format( - getattr(self, "VC_mode", -1), self._vc_fsm_log_snip(), t, led)) - except Exception: - pass - print("AI_key_release123") - self.key_state = 0 - if self.trans_flag == 1:#AI翻译模式 - EventMesh.publish("stop_voice_record") #关闭语音录音 - return - if self.voice_flag == 1: - self.stop_voice_timer.stop() - self.voice_flag=0 - if EventMesh.publish("GET_USB_STATE") == 0: - EventMesh.publish("set_led_state", {"ai_state": "idle"}) - EventMesh.publish("stop_voice_record")#停止静默录音 - tts_play_str = "留言已发送" - self.voice_flag=0 - EventMesh.publish("play_tts",[tts_play_str,1]) - return - if self.VC_mode == 1: - self._ai_key_release_handle_ptt() - return - def pwk_key_double_press(self,event,msg):#pwk双击 - print("come 双击") - # 切换模式：必须打断当前对话/录音/播放，回到 idle，等待按键触发新一轮 - try: - self.Close_flag = 1 - except Exception: - pass - try: - EventMesh.publish("publish_AI_btnInfo", {"func": self.Close}) - except Exception: - pass - if self.VC_mode == 0: - self.VC_mode = 1 - tts_str = "已切换到单对话模式" - # 下一次“无 dialog_id 的提示音 TTS”归类为模式切换提示（单对话） - try: - self._pending_tts_kind = "mode_switch_ptt" - self._last_tts_kind = "mode_switch_ptt" - except Exception: - pass - else: - self.VC_mode = 0 - tts_str = "已切换到长对话模式" - # 下一次“无 dialog_id 的提示音 TTS”归类为模式切换提示（长对话） - try: - self._pending_tts_kind = "mode_switch_long" - self._last_tts_kind = "mode_switch_long" - except Exception: - pass - # 提示音播完即退出（Close_flag=1），保持 idle，等待按键 - EventMesh.publish("play_tts",[tts_str,1]) - EventMesh.publish("set_storage_config",["VC_mode",self.VC_mode]) - - def AI_key_long_press(self, event, msg):#按键长按 - try: - if "vc_config" in globals() and vc_config and vc_config.get_bool("KEY_LED_DEBUG", False): - try: - led = EventMesh.publish("get_led_debug_state") - except Exception: - led = None - try: - t = getattr(self.fsm, "turn", None) - except Exception: - t = None - print("[KEY] AI_key_long_press VC_mode={} {} turn={} led={}".format( - getattr(self, "VC_mode", -1), self._vc_fsm_log_snip(), t, led)) - except Exception: - pass - print("AI_key_long_press") - self._ai_key_long_press_route(event, msg) - - def pwk_key_release(self, event, msg):#电源键短按，关闭 - try: - if "vc_config" in globals() and vc_config: - _m = bool(vc_config.get_bool("SINGLE_POWER_KEY_AI_MERGED", False)) - if not _m: - _ai = vc_config.get_attr("EXTINT_AI_GPIO", "GPIO22") - _m = _ai is None or _ai == "" - else: - _m = False - except Exception: - _m = False - if _m: - self._handle_pwk_short_release(event, msg, merged_ai_tail=True) - return - self._handle_pwk_short_release(event, msg, merged_ai_tail=False) - - - def key_prompt(self): - EventMesh.publish("audio_montage_play",["key_prompt.mp3"]) - #self.aud.play(1,1,'U:/static/key_prompt.mp3')#提示请您按住中间的按钮开始讲话，讲完松开即可 - def VC_error(self, event, msg):#错误播报 - self.websocket_error() - #EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_error}) - def set_VC_record_state_flag(self,data): - - self.VC_record_state_flag = data - print("[VC] record_state_flag={}".format(self.VC_record_state_flag)) - #开始录音，不缓存，直接发 - def VC_Speech_Start_now(self): - # 新一轮开始录音：允许上行（上一轮可能已置 silence 闸门） - self._server_vad_silence = 0 - self.VC_Speech_Start() - if self.VC_record_state_flag == 1: - self.VC_record_state_flag =2 - print("[VC] Speech_Start_now record_state={}".format(self.VC_record_state_flag)) - #开始录音，先缓存到buf，等服务器响应再发 - def VC_Speech_Start(self): - try: - - #self.VC_state = VC_State_SpeechStart - print("[VC] Speech_Start record_state={} AI_ASR_flag={}".format(self.VC_record_state_flag,self.AI_ASR_flag)) - if self.AI_ASR_flag != 0: - return - - if self.VC_record_state_flag == 0: - # 新一轮开始录音：允许上行（上一轮可能已置 silence 闸门） - self._server_vad_silence = 0 - self.AI_ASR_str = "" - self.AI_ASR_str_tmp = "" - self._last_asr_result_text = "" - try: - self._asr_completed_tone_done = 0 - except Exception: - pass - - self.VC_record_thread_close()#关闭录音线程 - _thread.stack_size(self.Audio_buf_size+8*1024) - self.VC_record_thread_id = _thread.start_new_thread(self.VC_record_thread,()) - _thread.stack_size(8*1024) - print("self.VC_record_thread_id:{}".format(self.VC_record_thread_id)) - try: - - if self.VC_record_TYPE == VC_record_TYPE_AMR : - try : - self.record.stream_start(audio.Record.AMRNB, self.VC_record_samplerate, 0) - except Exception as e: - usys.print_exception(e) - elif self.VC_record_TYPE == VC_record_TYPE_PCM: - print("record_pcm.stream_read_start()!!!!!!!!!!!!") - self.record.stream_start(audio.Record.WAV, self.VC_record_samplerate, 0) - # if self.record_pcm != None: - # try: - # self.record_pcm.stream_stop() - # self.record_pcm.close() - # self.record_pcm = None - # except Exception as e: - # usys.print_exception(e) - # if self.record_pcm == None: - # ret = self.record_pcm = audio.Audio.PCM(0,1,16000,0,1,2) - # print("audio.Audio.PCM:ret:{}".format(ret)) - # ret = self.record_pcm.stream_read_start() - # print("self.record_pcm.stream_read_start:ret:{}".format(ret)) - except Exception as e: - usys.print_exception(e) - - self.VC_record_state_flag = 1 - # 录音硬件已起：协议侧应对应 audio.create（PTT）或长对话直传 PCM - try: - print("[VC][proto] record_start sr={} pcm_type={} VC_mode={} session_started={} " - "ptt_audio_created={} thread_id={}".format( - self.VC_record_samplerate, self.VC_record_TYPE, self.VC_mode, - getattr(self, "_vc_session_started", -1), - getattr(self, "_ptt_audio_created", -1), - self.VC_record_thread_id)) - except Exception: - pass - # 录音开始：聆听灯（PTT 蓝快闪 / 长对话蓝慢闪） - try: - self._vc_publish_led("listening", "VC_Speech_Start") - except Exception: - pass - try: - if getattr(self, "VC_mode", -1) == 1: - self._ptt_on_record_hardware_started() - except Exception: - pass - - except Exception as e: - self.websocket_error() - self.VC_record_state_flag = 1 - self.VC_Speech_End()#关闭录音 - usys.print_exception(e) - #停止录音 - def VC_Speech_End(self): - try: - self._uplink_mark_end("VC_Speech_End") - self.AI_ASR_flag = 0 - self.AI_ASR_str_tmp = "" - # 勿在此清空 AI_ASR_str：vad.silence 停麦常早于 asr.completed，清空会导致 check_text:len:0 连播 think 音 - #EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_asr}) - - if self.VC_record_state_flag == 0: - return - #self.AI_client_asr_StopTranscription_requset()#请求停止 - if self.VC_record_TYPE == VC_record_TYPE_AMR: - self.record.stream_stop() - self.record.stop()#关闭录音 - Record_read_buf = bytearray(self.Audio_buf_size) - for i in range(10): - Record_len = self.record.stream_read(Record_read_buf, self.Audio_buf_size) - # if VC_AUDIO_FRAME_LOG: - # print("Record_len:!!!!!!{}".format(Record_len)) - if Record_len <= 0: - break - elif self.VC_record_TYPE == VC_record_TYPE_PCM: - self.record.stream_stop() - self.record.stop()#关闭录音 - Record_read_buf = bytearray(self.Audio_buf_size) - for i in range(10): - Record_len = self.record.stream_read(Record_read_buf, self.Audio_buf_size) - # if VC_AUDIO_FRAME_LOG: - # print("Record_len:!!!!!!{}".format(Record_len)) - if Record_len <= 0: - break - # if self.record_pcm != None: - # try: - # self.record_pcm.stream_stop() - # self.record_pcm.close() - # self.record_pcm = None - # except Exception as e: - # usys.print_exception(e) - except Exception as e: - usys.print_exception(e) - self.VC_record_thread_close()#关闭录音线程 - if self.AI_record_tmp_queue.empty() != True: - self.AI_record_tmp_queue.clear() - - self.VC_record_state_flag = 0 - # PTT 一轮录音结束：清长按语义，避免后续短按松手误走 end_speech 而不播提示音 - try: - if getattr(self, "VC_mode", -1) == 1: - self.AI_key_long_press_flag = 0 - except Exception: - pass - # 录音停止：单对话进入思考态（等服务器），勿继续蓝闪聆听 - try: - if getattr(self, "VC_mode", -1) == 1: - self._ptt_await_server = 1 - self._vc_publish_led("thinking", "VC_Speech_End/ptt") - else: - self._ptt_await_server = 0 - self._vc_publish_led("idle", "VC_Speech_End") - except Exception: - pass - print("[VC] Speech_End record_state={}".format(self.VC_record_state_flag)) - #关闭录音线程 - def VC_record_thread_close(self): - if self.VC_record_thread_id != 0: - count =50 - while True: - count -= 1 - if self.VC_record_thread_lock == 1: - self.VC_record_thread_lock = 2 - if self.VC_record_thread_lock == 0 or count <=0: - break - else: - print("VC_record_thread_thread going:{}...".format(count)) - utime.sleep_ms(1) - self.VC_record_thread_lock = 0 - print("VC_record_thread_thread_close") - if self.VC_record_thread_id != 0: - try: - _thread.stop_thread(self.VC_record_thread_id) - except Exception as e: - usys.print_exception(e) - self.VC_record_thread_id = 0 - #获取录音数据 - - def VC_record_thread(self): - ##return - print("[VC] record_thread start") - count = 0 - RequestToSpeak_flag = 0 - # try: - # uos.remove('/usr/test.pcm') - # except Exception as e: - # usys.print_exception(e) - #record_data_queue = Queue(100) - #fd = open('/usr/test.pcm','w') - # if self.VC_record_TYPE == 1: - # h_data= [0x23,0x21 ,0x41 ,0x4D ,0x52 ,0x0A ] - # h_data_b = bytes(h_data) - # self.AI_client_VC_send(h_data_b) - AI_record_tmp_queue_len = 0 - vad_buf = bytearray() - VAD_data_count = 0 - vad_state = 0 - self.VAD_count = 0 - while True: - try: - if self.VC_record_thread_lock == 2: - break - self.VC_record_thread_lock = 1 - # 死循环接收数据 - #if self.VC_state == VC_State_SpeechStart: - Record_read_buf = bytearray(self.Audio_buf_size) - - Record_len = 0 - if self.VC_record_TYPE == VC_record_TYPE_PCM:#pcm - Record_len = self.record.stream_read(Record_read_buf, self.Audio_buf_size) - #utime.sleep_ms(60) - - else : - - Record_len = self.record.stream_read(Record_read_buf, self.Audio_buf_size) - utime.sleep_ms(600) - - #Record_len = 1024*2 - #print("VC_record_thread:{}".format(Record_len)) - if Record_len > 0: - read_buf = bytes(Record_read_buf[:Record_len]) - # if VC_AUDIO_FRAME_LOG: - # print("Record_len:{}".format(Record_len)) - if self.record_data_save_flag == 1: - if self.record_fd == None: - try: - uos.remove('/usr/record_data.pcm') - except Exception as e: - pass - # 二进制写入，避免文本模式导致数据异常 - self.record_fd = open('/usr/record_data.pcm', 'wb') - self.record_fd_len = 0 - if self.record_fd != None: - try: - self.record_fd.write(read_buf) - print("write record:{}".format(len(read_buf))) - self.record_fd_len = self.record_fd_len+len(read_buf) - if self.record_fd_len >=500*1024: - EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) - except Exception as e: - usys.print_exception(e) - EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) - # count+=Record_len - # if count >150*1024: - # self.record_fd.close() - #self.record_fd = None - #VAD检测 - if self.is_VAD == 1: - vad_state = 0 - tmp_c = 1000 - while True: - tmp_c -=1 - if tmp_c <=0: - break - #vad_buf_len = len(vad_buf) - if VAD_data_count == 0:#没数据的时候 - vad_buf = Record_read_buf[:self.VAD_data_len] - VAD_data_count = self.VAD_data_len - self.vad.custom_vad_reset()#重置vad - else:#有数据的时候 - if Record_len - VAD_data_count < self.VAD_data_len :#剩余长度不够一次填充，先保留，但不填算法 - vad_buf = Record_read_buf[VAD_data_count:Record_len] - VAD_data_count = 0 - break - else:#剩余长度够填一次数据 - vad_buf = Record_read_buf[VAD_data_count:VAD_data_count+self.VAD_data_len] - VAD_data_count += len(vad_buf) - #print("vad_buf_len:{},{}".format(len(vad_buf),VAD_data_count)) - - if len(vad_buf) >= self.VAD_data_len: - vad_state = self.vad.custom_vad_process(vad_buf,len(vad_buf))#给vad数据 - #print("vad_state:{}".format(vad_state)) - - if VAD_data_count >= Record_len: - VAD_data_count = 0 - break - if vad_state == 1: - print("检测到VAD=1！！！！！！！！！！！！！") - - # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_RequestToSpeak})#打断 - # RequestToSpeak_flag = 1 - - self.VAD_count+=1 - if self.VAD_count >= 20: - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_RequestToSpeak})#打断 - - else: - if self.VAD_count != 0: - EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Play_Pause,"data":0})#恢复音频播放 - self.VAD_count = 0 - if self._vc_tts_pipeline_busy(): - pass - if vad_state == 1: - self.AI_record_tmp_queue.put(read_buf)#入缓存队列 - AI_record_tmp_queue_len += len(read_buf) - - - else: - VAD_data_count = 0 - if self.VC_record_state_flag <=1 :#没发送前先存起来 - self.AI_record_tmp_queue.put(read_buf) - AI_record_tmp_queue_len += len(read_buf) - else: - AI_record_tmp_queue_len = 0 - while True: - if self.AI_record_tmp_queue.empty() == True: - break - else: - pass - # 协议门控：会话未 started 前禁止上行音频（否则 session not initialized） - if VC_PROTO_GUARDS and self._vc_session_started != 1 and self.record_data_save_flag == 0: - _ = self.AI_record_tmp_queue.get() - AI_record_tmp_queue_len = max(0, AI_record_tmp_queue_len - len(_)) - continue - if not (self._server_vad_silence == 1 and self.record_data_save_flag == 0): - self.AI_client_VC_send(self.AI_record_tmp_queue.get()) #发送之前存的数据 - else: - self._uplink_mark_end("silence_gate(drop_queue)") - _ = self.AI_record_tmp_queue.get() - # 当前帧发送前协议门控（可用 VC_PROTO_GUARDS 临时关闭）： - # - session started 前禁止发帧 - # - Push-To-Talk 在 audio.create 前禁止发帧 - if VC_PROTO_GUARDS and ( - (self._vc_session_started != 1 and self.record_data_save_flag == 0) or - (self.VC_mode == 1 and self._ptt_audio_created != 1 and self.record_data_save_flag == 0) - ): - # 未 ready：转入缓存，等待 started 再发（并做容量控制） - try: - self.AI_record_tmp_queue.put(read_buf) - AI_record_tmp_queue_len += len(read_buf) - while AI_record_tmp_queue_len > 128*1024 and self.AI_record_tmp_queue.empty() != True: - d = self.AI_record_tmp_queue.get() - AI_record_tmp_queue_len = max(0, AI_record_tmp_queue_len - len(d)) - except Exception: - pass - else: - if not (self._server_vad_silence == 1 and self.record_data_save_flag == 0): - self.AI_client_VC_send(read_buf) - else: - self._uplink_mark_end("silence_gate(drop_buf)") - - - - - - - #print("VC_record_thread:{}".format(len(read_buf))) - #count+=len(read_buf) - # if count>=500*1024:#结束 - # self.AI_client_VC_end_speech() - # break - # count+=1 - # if count %10 == 0: - # print("VC_record_thread on line {} asr recv :{} send:{}".format(self.VC_record_state_flag,_thread.threadIsRunning(self.AI_client_asr_recv_id),_thread.threadIsRunning(self.AI_VC_send_data_queue_thread_id))) - - # if self.VC_state >= VC_State_SpeechEnd: - # break - - except Exception as e: - usys.print_exception(e) - #break - self.VC_record_thread_lock = 0 - if self.record_data_save_flag == 1: - utime.sleep_ms(1) - else: - utime.sleep_ms(100) - #fd.close() - print("VC_record_thread:exit") - self.VC_record_thread_id = 0#线程退出 - self.VC_record_thread_lock = 0 -###################################################### VC ################################################################################### - def websocket_connect_VC(self): - print("websocket_connect_VC:{} e:{}!!!!!!!!!!!!!!!!!!!!!".format(self.AI_client_VC_isconnect,self.AI_client_VC_debug)) - if self.AI_client_VC_isconnect == 1: - return - # 创建VC客户端 - try: - signal = net.csqQueryPoll() - if signal == -1 or sim.getStatus() != 1:#无蜂窝/无卡等：弱网提示 - self.websocket_error(play_weak_signal_tone=True) - return - - req_str = "" - ack = "" - if self.is_VC_ACK == 1: - ack = "ack=1" - lang = "lang=zh-cn"#普通话测试环境 - try: - if "lang" in self.VC_TTS_voice_type[self.VC_TTS_voice_key]: - lang = self.VC_TTS_voice_type[self.VC_TTS_voice_key]["lang"] - else: - print("not lang!!!!!!!!!!!!!!!!!!!!!!") - except Exception as e: - usys.print_exception(e) - - #女，男，童，粤，川 - self.bot_name = "柒先生" - try: - if "bot_name" in self.VC_TTS_voice_type[self.VC_TTS_voice_key]: - self.bot_name = self.VC_TTS_voice_type[self.VC_TTS_voice_key]["bot_name"] - else: - print("not bot_name!!!!!!!!!!!!!!!!!!!!!!") - except Exception as e: - usys.print_exception(e) - - # Headers = {'X-NLS-Token':'0c8b8f05a46f423096be64a9c4c46310'} - # self.AI_client_VC = uwebsocket.Client.connect('wss://nls-gateway-cn-beijing.aliyuncs.com/ws/v1',headers=Headers, debug=False) - token = EventMesh.publish("get_dev_token") - print("token:{}".format(token)) - - req_str = "vc/chat?is_auth={}".format(self.is_auth) - try: - if "vc_config" in globals() and vc_config: - base = vc_config.get_attr("VC_WS_PATH", "vc/chat") or "vc/chat" - req_str = "{}?is_auth={}".format(base, self.is_auth) - except Exception: - pass - if(token): - url = 'ws://{}/{}&dt={}'.format(self.server_ip,req_str,token) - else: - url = 'ws://{}/{}'.format(self.server_ip,req_str) - #url = 'ws://115.190.94.208:22080/vc/coze?dt={}'.format(token) - - if VC_WS_IO_LOG: - print("[VC] ws url {}".format(url)) - self.AI_client_VC = uwebsocket.Client.connect(url, debug=False) - #self.AI_WS_connect_Queue.put(self.AI_client_VC) - - - print("[VC] websocket_connect ok") - except Exception as e: #网络连接异常（已通过 CSQ/SIM，多为 URL/握手/服务端问题，勿播「信号弱」） - usys.print_exception(e) - print("websocket_connect_VC error!!!\r\n") - try: - print("[VC] ws connect exc: {}".format(e)) - except Exception: - pass - str_error = str(e) - if str_error.find("[Errno 22] EINVAL")!= -1: - print("[VC] websocket EINVAL: {}".format(str_error)) - self.websocket_error(play_weak_signal_tone=False) - EventMesh.publish("set_storage_config",["powerRestart",1])#调试临时关闭 - utime.sleep_ms(1000) - self.aud.stopPlayStream() - self.aud.stopAll() - Power.powerRestart() - - self.websocket_error(play_weak_signal_tone=False) - return - #print("AI_client_VC:{}".format(self.AI_client_VC)) - if self.AI_client_VC == None: - EventMesh.publish("update_dev_token")#获取token - self.websocket_error(play_weak_signal_tone=False) - return - self.AI_client_VC_debug+=1 - utime.sleep_ms(10) - self.AI_client_VC.settimeout(5) - #self.AI_VC_send_data_queue_thread_close()#关闭发送线程 - if self.AI_VC_send_data_queue_thread_id == 0: - _thread.stack_size(self.Audio_buf_size+8*1024) - self.AI_VC_send_data_queue_thread_id = _thread.start_new_thread(self.AI_VC_send_data_queue_thread, ()) - _thread.stack_size(8*1024) - print("AI_VC_start :AI_VC_send_data_queue_thread_id :{} ".format(self.AI_VC_send_data_queue_thread_id)) - self.AI_client_VC_recv_thread_close()#关闭读线程 - # 线程接收数据 - _thread.stack_size(self.Audio_buf_size+8*1024) - self.AI_client_VC_recv_id = _thread.start_new_thread(self.AI_client_VC_recv, (self.AI_client_VC,)) - _thread.stack_size(8*1024) - self.AI_client_VC_isconnect = 1 - print("websocket_connect_VC end") - - def websocket_error(self, TTS_str=None, play_weak_signal_tone=False): - """play_weak_signal_tone=True 时播 xinhaoruo1（弱网/无卡类）；连接/协议失败请传 False，避免误报「信号弱」。""" - try: - if getattr(self, "VC_mode", 0) != 1: - print("[VC][long_idle] websocket_error enter VC_mode={} TTS_str={} weak_hint={} -> Close then uplink_stop(async)".format( - getattr(self, "VC_mode", -1), TTS_str, play_weak_signal_tone)) - except Exception: - pass - print("websocket_error1!!!!") - utime.sleep_ms(800) - self.Close() - print("websocket_error1.1!!!!") - print("websocket_error!!!!") - print("websocket_error!!!!") - self.VC_Thinking_timerout = 0 - self.VC_satrt_flag = 0 - self.AI_TTS_str = "" - self.AI_ASR_str = "" - self.AI_ASR_str_tmp = "" - self.AI_ASR_flag = 0 - # 可能由收包线程/其它线程触发：统一切回主线程执行 stop，避免“异常→重试→再异常”的循环 - try: - if getattr(self, "VC_mode", 0) != 1: - print("[VC][long_idle] websocket_error schedule uplink_stop reason=websocket_error + set_led_idle") - except Exception: - pass - EventMesh.publish("publish_AI_btnInfo",{"func":self._uplink_request_stop, "data":"websocket_error"}) - EventMesh.publish("set_led_state", {"ai_state": "idle"}) - if TTS_str == None: - self.websocket_error_count+=1 - # if self.websocket_error_count <=3: - # EventMesh.publish("audio_montage_play",["xinhaoruo1.mp3"]) - if play_weak_signal_tone and self.websocket_error_count < 10: - EventMesh.publish("audio_montage_play", ["xinhaoruo1.mp3"]) - elif self.websocket_error_count >=10: - # EventMesh.publish("paly_iccid_imei") - EventMesh.publish("audio_montage_play", ["xinhaoruo1.mp3"]) - # else: - # EventMesh.publish("audio_montage_play",["R_SIM_FAILE.mp3"]) - #self.aud.play(1,1,'U:/static/R_SIM_FAILE.mp3') - if self.websocket_error_count <3 : - utime.sleep_ms(3000) - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - else: - print("websocket_error:tts:{}".format(TTS_str)) - self.AI_TTS_str = TTS_str - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - self.Close_flag = 1 - print("websocket_error2{}".format(self.websocket_error_count)) - - - print("websocket_error4") - - - def websocket_deconnect_VC(self): - #print("websocket_deconnect_VC {}!!!!!!!".format(self.AI_client_VC_isconnect)) - print("websocket_deconnect_VC!!!!!!!") - self._vc_session_started = 0 - # 断链/模式切换时必须重置“上行录音状态”，否则会被 _uplink_request_start 的幂等门控挡住， - # 表现为：切回长对话后无法自动开麦，按键 cancel/canceled 也无法恢复。 - try: - self.VC_record_state_flag = 0 - except Exception: - pass - try: - self._server_vad_silence = 0 - except Exception: - pass - try: - self._ptt_audio_created = 0 - except Exception: - pass - try: - # 若录音线程/录音设备仍在占用，先停掉 - self.VC_Speech_End() - except Exception: - try: - self.VC_record_thread_close() - except Exception: - pass - # 断链必须同步重置 FSM，否则 await_played/turn 残留会导致动态切换后“无法自动开麦/按键无效” - try: - self.fsm.session = VCSessionState.Disconnected - self.fsm.turn = VCTurnState.Idle - self.fsm.await_played = False - self.fsm.last_tts_completed_ms = 0 - self.fsm.last_llm_ms = 0 - self.fsm.reset_rx_phases() - except Exception: - pass - self.AI_client_VC_recv_thread_close()#关闭读线程 - self.AI_client_VC_recv_id = 0 - if self.AI_client_VC_isconnect == 1 or self.AI_client_VC!=None: - try: - self.AI_client_VC_isconnect = 0 - count = 500 - while True:#等待发完数据 - count-=1 - if count <=0 or self.AI_VC_send_data_queue_thread_lock == 0: - break - utime.sleep_ms(1) - - try: - self.AI_client_VC.close() - self.AI_client_VC_debug -=1 - #self.AI_client_VC=None - except Exception as e: - usys.print_exception(e) - try : - self.AI_client_VC.close() - self.AI_client_VC_debug -=1 - #self.AI_client_VC=None - except Exception as e: - usys.print_exception(e) - try : - self.AI_client_VC._close() - except Exception as e: - usys.print_exception(e) - self.AI_client_VC = None - except Exception as e: - usys.print_exception(e) - print("websocket_deconnect_VC {}!!!!!!!".format(self.AI_client_VC_isconnect)) - #websocket发送ping - def AI_client_VC_ping(self): - pass - if self.AI_client_VC_isconnect == 1: - #self.AI_client_VC.write_frame(0x09,b'ping ping') - self.AI_client_VC_send("ping") - if VC_HEARTBEAT_LOG: - print("VC: ping ping") - - #websocket发送数据 - def AI_client_VC_send(self,data_str): - # if isinstance(data_str, str):#是字符串 - # print("AI_client_VC_send:{}".format(data_str)) - # pass - # else: - # print("AI_client_VC_send:len{}".format(len(data_str))) - # 统计“语音上行”真实开始/结束：这里只统计二进制音频（非 ping/字符串） - try: - if not isinstance(data_str, str): - l = len(data_str) - if l > 0: - self._uplink_mark_start(l) - self._uplink_bytes += l - except Exception: - pass - self.AI_VC_send_data_queue.put([AI_client_VC_num,data_str]) - #关闭数据发送线程 - def AI_VC_send_data_queue_thread_close(self): - if self.AI_VC_send_data_queue_thread_id != 0: - # 先清空待发队列，避免关闭过程持续堆积/刷屏 - if self.AI_VC_send_data_queue.empty() != True: - self.AI_VC_send_data_queue.clear() - # 通知发送线程退出；若发送线程正阻塞在底层 send，则只能等超时或强制 stop_thread - self.AI_VC_send_data_queue_thread_lock = 2 - count = 500 - while True: - count -= 1 - if self.AI_VC_send_data_queue_thread_lock == 0 or count <=0: - break - else: - # 该打印只是“等待线程停下来”的调试心跳，默认关闭以免影响系统实时性 - if VC_HEARTBEAT_LOG and (count % 50 == 0 or count > 490): - print("AI_VC_send_data_queue_thread going:{}...".format(count)) - utime.sleep_ms(5) - self.AI_VC_send_data_queue_thread_lock = 0 - if VC_HEARTBEAT_LOG: - print("AI_VC_send_data_queue_thread_close") - if self.AI_VC_send_data_queue_thread_id != 0: - try: - # 线程可能已自然退出/被复用，stop_thread 会抛 ValueError - try: - running = _thread.threadIsRunning(self.AI_VC_send_data_queue_thread_id) - except Exception: - running = True - if running: - _thread.stop_thread(self.AI_VC_send_data_queue_thread_id) - except Exception as e: - usys.print_exception(e) - self.AI_VC_send_data_queue_thread_id = 0 - #发送数据队列 - def AI_VC_send_data_queue_thread(self): - print("AI_VC_send_data_queue_thread:start") - count_t = 0 - while True: - if self.AI_VC_send_data_queue_thread_lock == 2: - print("AI_VC_send_data_queue_thread,该退出了") - break - if self.AI_VC_send_data_queue.empty() :#不能去掉，去掉会卡死 - self.AI_VC_send_data_queue_thread_lock = 0 - count_t+=1 - # if count_t%200 == 0: - # print("self.AI_VC_send_data_queue.empty") - utime.sleep_ms(1) - else: - #data[0]是websocket连接对象编号，data[1]是数据 - #print("AI_VC_send_data_queue_thread:get") - - self.AI_VC_send_data_queue_thread_lock = 1 - data = self.AI_VC_send_data_queue.get() - - - #print("AI_VC_send_data_queue_thread:num{} len{}".format(data[0],len(data[1]))) - if data != None: - try: - #data["func"](data["data"]) - # 发送数据 - #bytes_data = bytes(data[1]) - #data[0].send(data[1]) - #print("AI_VC_send_data_queue_thread:len:{}".format(len(data[1]))) - #print("AI_VC_send_data_queue_thread on line asr recv :{} vc recv:{}".format(_thread.threadIsRunning(self.AI_client_asr_recv_id),_thread.threadIsRunning(self.AI_client_VC_recv_id))) - if data[0] == AI_client_VC_num: - - if isinstance(data[1], str):#是字符串 - self.VC_pingping_timerout = 0 - s = data[1] - if s is None: - s = "" - # 保活 ping 不落 WS IO 日志（避免周期性刷屏） - if VC_WS_IO_LOG and s != "ping": - try: - if len(s) > 240: - preview = s[:160] + " ... " + s[-60:] - else: - preview = s - print("[VC] WS send_txt len={} txt={}".format(len(s), preview)) - except Exception: - try: - print("[VC] WS send_txt {}".format(data[1])) - except Exception: - pass - elif not VC_WS_IO_LOG: - if s == "ping": - pass - else: - try: - ev, tid = _vc_out_json_event_meta(s) - if ev: - print("[VC][proto] tx event={} trace_id={} len={} ws_conn={}".format( - ev, tid or "-", len(s), self.AI_client_VC_isconnect)) - else: - print("[VC][proto] tx unknown_json len={} ws_conn={}".format( - len(s), self.AI_client_VC_isconnect)) - except Exception: - print("[VC][proto] tx parse_fail len={}".format(len(s))) - else: - pass - # if VC_AUDIO_FRAME_LOG: - # print("VC: send_data len:{}".format(len(data[1]))) - # print("self.AI_client_VC = {} , self.AI_client_VC_isconnect = {}".format(self.AI_client_VC,self.AI_client_VC_isconnect)) - if self.AI_client_VC != None and self.AI_client_VC_isconnect == 1: - self.AI_client_VC.send(data[1]) - else: - try: - if isinstance(data[1], str) and data[1] != "ping": - _ev, _tid = _vc_out_json_event_meta(data[1]) - print("VC VC: send_data:ERROR len={} event={} trace_id={}".format( - len(data[1]), _ev or "?", _tid or "-")) - else: - print("VC VC: send_data:ERROR:{}".format(len(data[1]))) - except Exception: - print("VC VC: send_data:ERROR:{}".format(len(data[1]))) - #print("VC: send_data end") - - #print("VC: send_data end") - except Exception as e: - print("DeviceInfo_queue_thread:ERROR:{}".format(e)) - usys.print_exception(e) - if data[0] == AI_client_VC_num: - #self.websocket_error() - pass - - - try: - if self.AI_client_VC_isconnect != 0: - print("VC发送数据错误，退出") - EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_error,"data":"VC发送出错了，请重试"}) - self.ud_error(["VC",self.VC_taskid]) - except Exception as e: #网络连接异常 - usys.print_exception(e) - - utime.sleep_ms(1) - self.AI_VC_send_data_queue_thread_lock = 0 - #VC关闭读线程 - def AI_client_VC_recv_thread_close(self): - if self.AI_client_VC_recv_id != 0: - count =500 - while True: - count -= 1 - if self.AI_client_VC_recv_lock == 1: - self.AI_client_VC_recv_lock = 2 - if self.AI_client_VC_recv_lock == 0 or count <=0: - break - else: - print("AI_client_VC_recv_thread going:{}...".format(count)) - utime.sleep_ms(1) - self.AI_client_VC_recv_lock = 0 - print("AI_client_VC_recv_thread_close") - if self.AI_client_VC_recv_id!= 0: - try: - _thread.stop_thread(self.AI_client_VC_recv_id) - except Exception as e: - usys.print_exception(e) - self.AI_client_VC_recv_id = 0 - #Ai VC websocket返回数据 - def AI_client_VC_recv(self,cli): - count = 0 - asr_led_flag = 0 - tts_led_flag = 0 - while True: - try: - # 死循环接收数据 - self.AI_client_VC_recv_lock = 0 - - count+=1 - if self._vc_tts_pipeline_busy() and (self.tts_in_data_len - self.tts_out_data_len > 64*1024 or count>5): - count = 0 - utime.sleep_ms(1) - if not self._vc_tts_pipeline_busy(): - utime.sleep_ms(1) - recv_data = cli.recv() - if self.AI_client_VC_recv_lock == 2: - print("AI_client_VC_recv 该退出了") - break - self.AI_client_VC_recv_lock = 1 - if self._vc_rx_resets_long_idle_timer(recv_data): - self.close_timerout = 0 - if recv_data == "websocket close": - # 服务器关闭连接 - print("VC 服务器关闭连接!!!!!!!!!!!!!!!!!!!!!!!!!!!") - #self.AI_client_VC_isconnect = 0 - #self.websocket_deconnect_VC() - #EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_asr}) - # #EventMesh.publish("publish_AI_btnInfo",{"func":utime.sleep_ms,'data':(1500)}) - #EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_connect_asr}) - EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_error,"data":"VC连接断开了，请重试"}) - self.ud_error(["VC",self.VC_taskid]) - break - elif recv_data == "":#客户端关闭连接 - print("VC VC :客户端关闭连接") - EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_error,"data":"VC连接断开了，请重试"}) - self.ud_error(["VC",self.VC_taskid]) - break - try: - self.websocket_error_count = 0 - if isinstance(recv_data, str):#返回的是字符串 - jsload_data = ujson.loads(recv_data) - if VC_WS_IO_LOG: - print("[VC] recv raw:{}".format(recv_data)) - else: - try: - _rev = jsload_data.get("event", "") - if _rev != "llm.delta": - print("[VC] rx event={} sid={}".format( - _rev, jsload_data.get("sid", "") or "-")) - except Exception: - pass - #print("AI_client_VC_recv jsload_data = {}".format(jsload_data)) - try: - self._cur_sid = jsload_data.get("sid", "") or "" - except Exception: - self._cur_sid = "" - try: - ev = jsload_data.get("event", "") - if ev: - self._vc_dispatch_server_event(ev, jsload_data) - except Exception: - pass - - if jsload_data["event"] == "started" :#会话开始 - #payload = jsload_data["payload"] - # self.VC_dialog_id = payload["dialog_id"] - # self.VC_task_id = jsload_data["task_id"] - # started 仅表示链路就绪：不在这里提前切灯态； - # 聆听灯只在“录音真正开始/停止”处更新（VC_Speech_Start/End）。 - # 会话开始：允许上行；连续对话模式下直接进入“可发送音频”状态 - self._vc_session_started = 1 - # started 时不要强制清 Close_flag： - # - Close_flag==1：提示音/模式切换播完退出 - # - Close_flag==0：正常对话流程 - self._ptt_audio_created = 0 - self._server_vad_silence = 0 - self._turn_has_asr_completed = 0 - self._turn_has_vad_silence = 0 - self._turn_has_tts_started = 0 - self._llm_completed_ms = 0 - if self.VC_mode == 0 and getattr(self, "Close_flag", 0) == 0: - _skip_uplink = 0 - try: - if getattr(self, "_skip_started_uplink_once", 0) == 1: - self._skip_started_uplink_once = 0 - _skip_uplink = 1 - print("[VC][uplink_ctrl] Deny START by server.started cause=boot_prompt") - except Exception: - pass - if not _skip_uplink: - # 会话已 started：若此时上行未启动，则在这里补一次开麦，避免“Start 前开麦被 Close 停掉”后 - # 出现首轮无音频上行导致“第一次对话没有”的情况。 - if self.record_data_save_flag == 0: - if self.VC_record_state_flag == 0: - self.request_uplink_start("server.started") - else: - self._vc_clear_stale_record_state("server.started") - self.request_uplink_start("server.started/retry") - - print("VC:Started!!!!!!!!\r\n") - print("TTS音频为：{}，语音下发地址为：{}".format(self.AI_TTS_str,self.AI_new_voice_url)) - if self.AI_new_voice_url != "" and self.alarm_play_flag == 1 :#有闹钟,先响闹钟 - self.AI_VC_start() - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放铃声 - self.alarm_play_flag = 0 - self.voice_choose_flag = 1 - elif self.AI_TTS_str == "" and self.AI_new_voice_url != "" and self.music_play_flag == 0: - # if EventMesh.publish("get_history_voice_flag") == 1: - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 - # self.recv_voice = 0#当留言列表为空，留言播放标志清空 - # else: - # self.voice_cnt = 0 - # print("定时器开启222222222") - # self.voice_timer.start(1000,1,self.voice_judge) - elif self.AI_TTS_str != "" and self.AI_new_voice_url == "": - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_synthesis}) - - elif self.AI_TTS_str != "" and self.AI_new_voice_url != "" and self.music_play_flag == 0:#遇到tts和语音留言冲突时(并且没有音乐播放) - if EventMesh.publish("get_history_voice_flag") == 1:#播放历史留言 - if EventMesh.publish("get_wait_voice_flag") == 0: - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_synthesis}) - self.his_cnt = 0 - self.his_timer.start(1000,1,self.history_play) - self.voice_choose_flag = 1 - else:#等待留言 - if self.pwk_notice_flag == 0: - self.wait_cnt = 0 - print("未听播放1") - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_synthesis}) - self.wait_timer.start(1000,1,self.wait_play) - self.voice_choose_flag = 1 - - else:#已确定播放 - print("未听播放2") - self.wait_cnt = 0 - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_synthesis}) - self.wait_timer.start(1000,1,self.wait_single_play) - self.voice_choose_flag = 1 - else:#播放正常留言 - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_synthesis}) - self.voice_cnt = 0 - print("定时器开启111111111") - self.voice_timer.start(1000,1,self.voice_judge) - self.voice_choose_flag = 1 - # self.AI_TTS_str = "" - elif self.AI_TTS_str != "" and self.AI_new_voice_url != "" and self.music_play_flag == 1 :#遇到tts和语音留言冲突时(并且有音乐播放) - self.music_play_flag = 0 - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_synthesis}) - self.music_timer.start(1000,1,self.music_play) - self.voice_choose_flag = 1 - else: - #开始上传语音 - try: - if "vc_config" in globals() and vc_config and vc_config.get_bool("UPLOAD_START_TONE_ENABLED", False): - self.chenggong_tone() - except Exception: - pass - if self.VC_mode == 0: - self.request_uplink_start("server.started/idle") - else: - # Push-To-Talk：必须先同步发出 audio.create，再开麦。 - # request_send_audio_create() èµ° EventMesh 异步队列，会导致录音线程先于 - # audio.create 上行，深度睡眠唤醒后首轮 ASR 空结果、第二轮才正常。 - try: - self.AI_client_VC_send_speech() - except Exception: - self.request_send_audio_create() - # 若长按发生在“未 started”窗口，收到 started 后立即开麦进入新一轮 PTT - try: - if getattr(self, "_ptt_force_record_on_started", 0) == 1: - self._ptt_force_record_on_started = 0 - self.VC_Speech_Start_now() - except Exception: - pass - - - # if self.AI_new_voice_url != "": - # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice}) - - elif jsload_data["event"] == "config.update" :#配置更新中，需要停止录音 - try: - self.fsm.reset_rx_phases() - except Exception: - pass - self.request_uplink_stop("config.update") - - elif jsload_data["event"] == "asr.started" :#asr开始 - try: - self.fsm.on_server_asr_started() - except Exception: - pass - self._server_vad_silence = 0 - self._turn_has_asr_completed = 0 - self._turn_has_vad_silence = 0 - self._turn_has_tts_started = 0 - self._llm_completed_ms = 0 - try: - self._asr_completed_tone_done = 0 - except Exception: - pass - # 单对话：_drop_tts_audio 在 VC_Speech_Start 真开录后再清 0，避免 canceled 后仍播旧 TTS - try: - if getattr(self, "VC_mode", 0) != 1: - self._drop_tts_audio = 0 - except Exception: - pass - # 单对话：勿在 asr.started ç½® flag=2（未真开麦会僵死）；由 VC_Speech_Start 维护 - self.LocalRespondingEnded_timerout = 0 - elif jsload_data["event"] == "asr.result" :#asr结果 - try: - self.AI_ASR_str = jsload_data["payload"]["content"] - except Exception: - try: - self.AI_ASR_str = jsload_data.get("payload", {}).get("text", "") or "" - except Exception: - self.AI_ASR_str = "" - print("[VC][ASR] {}".format(self.AI_ASR_str)) - try: - if (self.AI_ASR_str or "").strip(): - self._last_asr_result_text = self.AI_ASR_str - except Exception: - pass - if self.AI_ASR_str == "":#没识别到 - self.AI_TTS_str = "我没听清，请再说一遍" - self._vc_led_alert_rb_flash() - elif jsload_data["event"] == "asr.completed" :#asr 终止的标识 - self._turn_has_asr_completed = 1 - try: - if not (self.AI_ASR_str or "").strip(): - self.AI_ASR_str = getattr(self, "_last_asr_result_text", "") or "" - except Exception: - pass - # 本地 think 提示：仅在有有效 ASR 文本时播，避免空轮次连播「滴」 - try: - if getattr(self, "_asr_completed_tone_done", 0) == 1: - pass - elif not self._vc_effective_asr_text(): - try: - print("[VC][ASR] completed empty, skip think tone") - except Exception: - pass - else: - stream_on = False - try: - stream_on = EventMesh.publish("get_vc_tts_playback_active") == 1 - except Exception: - stream_on = False - pcm_busy = getattr(self, "play_pcm", None) is not None - playing = getattr(self, "VC_Play_flag", 0) == 1 - try: - thinking = getattr(self.fsm, "turn", VCTurnState.Idle) == VCTurnState.Thinking - except Exception: - thinking = False - if (not stream_on) and (not pcm_busy) and (not playing) and (not thinking): - self._vc_local_play('U:/static/think.mp3', 1, 1, 650) - self._asr_completed_tone_done = 1 - except Exception: - pass - print("self.commands_mode:{}".format(self.commands_mode)) - if self.commands_mode ==2: - ret= self.check_text(self._vc_effective_asr_text()) - if ret == 0:#配置指令 - - if self.AI_VC_send_data_queue.empty()!=True: - self.AI_VC_send_data_queue.clear() - if self.AI_ASR_send_data_queue.empty()!=True: - self.AI_ASR_send_data_queue.clear() - self.AI_ASR_send_buf_len = 0 - EventMesh.publish("set_led_state", {"ai_state": "thinking"}) - try: - - self.request_uplink_stop("asr.completed/config_cmd") - EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - except Exception as e: - usys.print_exception(e) - continue - elif ret == 1:#不处理 - pass - #self.VC_Speech_End() - #EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) - - # if self.record_data_save_flag == 1: - # if self.record_fd : - # self.record_fd.close() - # self.record_fd = None - - # if dev_type != "7PIN" : - #EventMesh.publish("publish_AI_btnInfo",{"func":self.send_tone}) - elif jsload_data["event"] == "vad.active" :#VAD 检测到说话→聆听 - try: - if "vc_config" in globals() and vc_config and vc_config.get_bool("VAD_ACTIVE_BLUE_LED", True): - self._server_vad_silence = 0 - except Exception: - pass - elif jsload_data["event"] == "vad.silence" :#VAD 终止的标识 - # 说明服务端已判定“本轮说话结束”；此时继续上行只会干扰后续（例如推迟 TTS）。 - # 仅在“正在录音上行”的语音阶段才停止，避免误触发。 - self._server_vad_silence = 1 - self._turn_has_vad_silence = 1 - self._uplink_mark_end("vad.silence") - if self.record_data_save_flag == 0: - if self.VC_record_state_flag != 0 and self._vc_vad_uplink_stop_eligible(): - self.request_uplink_stop("vad.silence") - elif jsload_data["event"] == "llm.created" :#思考（LLM） - try: - self._ptt_await_server = 0 - except Exception: - pass - try: - EventMesh.publish("set_led_state", {"ai_state": "thinking"}) - except Exception: - pass - self._llm_completed_ms = 0 - # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Play_Start}) - pass - elif jsload_data["event"] == "llm.completed" :#llm完成（可能无 tts） - try: - self._llm_completed_ms = utime.ticks_ms() - except Exception: - # 勿用 1：ticks_diff(now,1)≈开机至今，会瞬间满足 thinking 超时误以为该重启 - self._llm_completed_ms = 0 - try: - self._cur_dialog_id = jsload_data.get("payload", {}).get("dialog_id", "") or "" - except Exception: - self._cur_dialog_id = "" - try: - self.fsm.log("server.llm.completed") - except Exception: - pass - pass - elif jsload_data["event"] == "llm.delta" :#llm返回数据 - if self.is_VAD == 0 and self.record_data_save_flag == 0: - if self.VC_record_state_flag == 1:#llm返回数据还没停止，就先停止 - self.request_uplink_stop("llm.delta") - elif jsload_data["event"] == "tts.started" :#tts开始 - try: - self._ptt_await_server = 0 - except Exception: - pass - self._server_vad_silence = 1 - self._turn_has_tts_started = 1 - self._tts_pending = 0 - self._tts_pending_ms = 0 - self._uplink_mark_end("tts.started") - # stop uplink + play_start 由 FSM action list 统一触发，避免多处重复调用 - # 标记本轮 TTS 来源： - # - 有 payload.dialog_id：认为是“对话回复 TTS” - # - 无 dialog_id：认为是“提示/提醒 TTS”（模式切换可通过 _pending_tts_kind 标记） - try: - did = jsload_data.get("payload", {}).get("dialog_id", "") or "" - except Exception: - did = "" - # 注意：无 dialog_id 的 tts.started 很可能是提示音/提醒音； - # 但“模式切换提示音”的 kind 可能已在 synthesis 时写入 _last_tts_kind， - # 这里不应无条件覆盖为 prompt，否则播完无法触发下一轮开麦。 - try: - if did: - self._last_tts_kind = "dialog" - else: - k = getattr(self, "_pending_tts_kind", "") or "" - if k: - self._last_tts_kind = k - else: - cur = getattr(self, "_last_tts_kind", "") or "" - if cur in ( - "mode_switch_long", - "mode_switch_ptt", - "boot_prompt", - ): - pass - elif not cur: - self._last_tts_kind = "prompt" - except Exception: - pass - try: - self._pending_tts_kind = "" - except Exception: - pass - # 播放灯由阶段事件控制 - elif jsload_data["event"] == "tts.completed" :#tts结束 - # tts.completed 仅表示下发结束，本地可能仍在播放；绿灯关闭在本地播放结束时处理 - try: - # server event 已在统一入口 on_server_event 触发 FSM，这里避免重复驱动 - pass - except Exception: - pass - try: - try: - sid = getattr(self, "_cur_sid", "") - did = getattr(self, "_cur_dialog_id", "") - except Exception: - sid, did = "", "" - print("[TTS] completed in_len={} out_len={} q_empty={} play_pcm={} sid={} did={}".format( - getattr(self, "tts_in_data_len", 0), - getattr(self, "tts_out_data_len", 0), - 1 if self.AI_audio_queue.empty() else 0, - 1 if self.play_pcm != None else 0, - sid, - did, - )) - except Exception: - pass - # 播完上报完全由 FSM await_played + play_end 驱动，这里不再使用 LocalRespondingEnded_timerout/_need_audio_played_ack - pass - - elif jsload_data["event"] == "canceled" :#打断 - # 停播/清队列已在 _vc_dispatch_server_event(canceled) 完成；此处仅收尾标志 - self.is_ruppt_tts = 0#设备上发中断tts，收到服务器返回的cancel后置为0 - self._need_audio_played_ack = 0 - self._tts_pending = 0 - self._tts_pending_ms = 0 - # 开麦由 FSM action list 统一触发（uplink_start:canceled），避免重复调用 - if self.AI_audio_queue_thread_id ==0: - _thread.stack_size(24*1024) - self.AI_audio_queue_thread_id = _thread.start_new_thread(self.AI_audio_queue_thread, ()) - _thread.stack_size(8*1024) - print("AI_VC_start :AI_audio_queue_thread_id :{} ".format(self.AI_audio_queue_thread_id)) - else: - print("AI_VC_start :AI_audio_queue_thread_id != 0 ") - elif jsload_data["event"] == "completed" :#会话结束（可能无 TTS，如 ASR 空） - # 重要：服务端常在 tts.completed 后紧跟 completed。 - # 若本轮存在 TTS/待发 audio.played，则不能在此清零 LocalRespondingEnded_timerout/_need_audio_played_ack， - # 否则会掐断「播完→audio.played→下一轮录音」链路，表现为最多 1~2 轮就停。 - if self._need_audio_played_ack == 1 or self._vc_tts_pipeline_busy() or self.VC_Play_flag == 1: - pass - else: - # 无 TTS 的 completed：清状态并按需进入下一轮（见下方条件） - self.LocalRespondingEnded_timerout = 0 - self._need_audio_played_ack = 0 - try: - self.fsm.reset_rx_phases() - except Exception: - pass - # 连续对话：若本轮没有 TTS（或服务端直接 completed），需要立刻回到聆听状态 - # 否则会出现“有时 1 è½®/2 轮后就停住”的现象（服务端 completed 但端侧未重新开录音上行）。 - if self.VC_mode == 0 and self.Close_flag == 0: - # 仅当“本轮已结束一次语句（vad.silence 或 asr.completed）且从未进入 tts.started” - # 才自动进入下一轮；避免把真正的“会话结束/异常结束”误当成下一轮入口。 - if (self._turn_has_tts_started == 0) and (self._turn_has_vad_silence == 1 or self._turn_has_asr_completed == 1): - self._server_vad_silence = 0 - # 开麦由 FSM action list 统一触发（uplink_start:completed），这里不再重复调用 - pass - - elif jsload_data["event"] == "command" :#命令匹配 - ret = -1 - try: - if self.commands_mode ==1: - if len(jsload_data["payload"]["functions"])>0: - - cmd = jsload_data["payload"]["functions"][0]["function"]["arguments"] - print("llm:cmd:{}".format(cmd)) - cmd = ujson.loads(cmd) - ret = self.check_text_llm(cmd) - elif self.commands_mode ==0: - - cmd = jsload_data["payload"] - print("server:cmd:{}".format(cmd)) - ret = self.check_text_server(cmd) - else: - cmd = jsload_data["payload"]["functions"][0]["function"]["arguments"] - print("llm:cmd:{}".format(cmd)) - cmd = ujson.loads(cmd) - ret = self.check_text_llm(cmd) - except Exception as e: - usys.print_exception(e) - if ret == -1: - if self.AI_TTS_str =="": - self.AI_TTS_str = "抱歉，没找到对应操作，可以换个说法试试" - #EventMesh.publish("publish_AI_btnInfo",{"func":utime.sleep_ms,'data':(10)}) - self.request_uplink_stop("command") - EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - elif jsload_data["event"] == "failed":#接口返回错误 - if jsload_data["payload"]["error"] == "concurrency_limit_exceeded": - EventMesh.publish("publish_AI_btnInfo",{"func":self.concurrency_limit_exceeded_tone}) - - #utime.sleep_ms(4000) - - else: - EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_error,"data":jsload_data["payload"]["message"]}) - pass - - elif jsload_data["event"] == "audio.convert.completed":#音频转成完成没有出错 接口 - # self.VC_state = VC_State_End - print("未读留言初始数值：{}".format(self.pwk_cnt_voice)) - if self.pwk_notice_flag == 1:#只有在播放的时候才会去计数 - if self.pwk_cnt_voice > 0: - self.pwk_cnt_voice -= 1#减1 - print("未读留言又-1,当前留言剩余数量为:{}".format(self.pwk_cnt_voice)) - else: - print("未读列表为空") - if self.pwk_cnt_voice == 0:#积累的留言已经全部播完 - self.pwk_notice_flag = 0 - self.wait_sort = 0 - self.wait_first_flag = 0#让下次未读播放提示恢复 - self.voicerupt_flag = 0 #标志位置0 - self.wait_voicerupt_flag = 0 - self.voice_notice = 1#开启回复 - if self.history_voice_flag == 1 and self.wait_voice_flag == 0: - self.voice_notice = 0 - EventMesh.publish("set_storage_config",["is_noread_voice",0]) - - if self.voice_choose_flag == 1:#播放完语音后，有冲突的时候关闭 - self.voice_choose_flag = 0 - self.Close_flag = 1 - self.LocalRespondingEnded_timerout = 8 - # EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) - if self.voice_notice == 1:#已收到留言，进行该留言回复逻辑处理 - tts_play_str = "如需回复，长按确认键开始录制，松开发送。十秒内无操作自动退出" - EventMesh.publish("play_tts",[tts_play_str,1]) - self.voice_notice = 0#标志位清空 - self.voice_timer.stop() - self.revoice_cnt = 0 - self.voice_flag = 1 - print("语音留言回复停止") - self.revoice_timer.start(1000,1,self.revoice_judge) - print("是否未读：{},是否播完：{}".format(EventMesh.publish("get_wait_voice_flag"),len(EventMesh.publish("get_play_new_voice_list")))) - # if EventMesh.publish("get_wait_voice_flag") == 1 and len(EventMesh.publish("get_play_new_voice_list")) ==0:#还在未读并且列表为空 - # self.wait_first_flag = 0 - # print("未读留言播完，清空标志") - # EventMesh.publish("set_wait_voice_flag",0) - # EventMesh.publish("set_history_voice_flag",0) - EventMesh.publish("update_play_new_voice_info")#更新已读状态 - self.recv_voice = 0 - print("不开启定时器标志位置0") - self.LocalRespondingEnded_timerout = 8 - try: - self.fsm.reset_rx_phases() - except Exception: - pass - - pass - elif jsload_data["event"] == "audio.convert.failed": #音频转换失败了 - pass - - - else:#下发音频数据 - #self.AI_ASR_str = ""#下发音频后清掉asr结果 - - - # PTT 打断后丢弃残留 TTS 音频帧，避免“断断续续继续播” - try: - if self._ptt_should_drop_inbound_tts(): - continue - except Exception: - pass - if self.VC_stop_play_audio_flag == 0: - print("新的AI对话不允许以前的播放数据:") - break - - # if VC_AUDIO_FRAME_LOG: - # print("下发音频数据1:{}".format(len(recv_data))) - - if self.play_data_save_flag == 1: - if self.play_fd ==None: - try: - uos.remove('/usr/play_data.pcm') - except Exception as e: - usys.print_exception(e) - self.play_fd = open('/usr/play_data.pcm','w') - if self.play_fd !=None: - self.play_fd.write(recv_data) - # print("下发音频数据2:{}".format(len(recv_data))) - #if self.VC_state >= VC_State_TTSStart: - #self.put_record_data([recv_data,""]) - if self.is_ruppt_tts == 0:#被设备主动打断打断则不播放，知道收到服务器返回的cancel才修改标志位 - if self.VC_play_TYPE == VC_play_TYPE_PCM: - if self.play_pcm != None: - try: - #print("play_pcm.stream_write !!!") - buf_len = self.play_pcm.stream_get_buf_len() - count_tmp = 1000 - while buf_len < len(recv_data): - utime.sleep_ms(1) - count_tmp-=1 - if count_tmp<=0 or self.play_pcm == None: - break - buf_len = self.play_pcm.stream_get_buf_len() - #print("buf已满，等待释放：buf_len:{},count_tmp:{}".format(buf_len,count_tmp)) - except Exception as e: - usys.print_exception(e) - EventMesh.publish("publish_AI_btnInfo",{"func":self.put_record_data,"data":[recv_data,""]}) - else: - print("被打断不播放") - utime.sleep_ms(1) - except Exception as e: - usys.print_exception(e) - - except Exception as e: - error_str = str(e) - #print("VC VC:e:{}".format(error_str)) - if error_str.find("[Errno 110] ETIMEDOUT")!= -1 or error_str.find("[Errno 115] ")!= -1: - if VC_HEARTBEAT_LOG: - print("VC VC:recv 正常报错：{}".format(error_str)) - # recv 超时仅表示本周期无帧，链路仍可能在等 LLM/TTS；清零避免误触发「接口无响应」 - self.close_timerout = 0 - pass - else: - usys.print_exception(e) - #self.websocket_error() - print("VC VC :读数据报错，退出") - EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_error,"data":"VC出错了，请重试"}) - self.ud_error(["VC",self.VC_taskid]) - break - self.AI_client_VC_isconnect = 0 - self.AI_client_VC_recv_id = 0 - self.AI_client_VC_recv_lock = 0 - print("[VC] recv_thread exit") - - def voice_play_10S(self,event,msg):#按键开启留言播放等待10S定时器判断 - self.voice_timer.start(1000,1,self.voice_judge) - #语音留言定时器 - def voice_judge(self, *args, **kwargs): - EventMesh.publish("publish_AI_btnInfo",{"func":self.voice_judger_cb}) - def voice_judger_cb(self): - self.voice_cnt += 1 - # print("语音留言定时器cnt = {}".format(self.voice_cnt)) - if self.voicerupt_flag == 1:#按键被按下 - self.pwk_notice_flag = 1#表示进入收听未听留言 - self.AI_VC_start() - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 - self.voice_cnt = 0 - # self.recv_voice = 0#进入留言定时器，留言清空 - print("进入留言定时器1，留言清空！！！！！") - EventMesh.publish("update_play_new_voice_info")#更新已读状态 - self.voice_timer.stop() - if self.pwk_cnt_voice > 0: - EventMesh.publish("get_new_voice_msg_S") - if self.voice_cnt > 15:#10S后退出播放,5S是为了给语音提示留时间 - self.voice_cnt = 0 - self.recv_voice = 0#进入留言定时器，留言清空 - if self.voice_choose_flag == 1:#播放完语音后，有冲突的时候关闭 - self.voice_choose_flag = 0 - print("进入留言定时器2，留言清空！！！！！") - self.voice_timer.stop() - #语音留言回复定时器 - def revoice_judge(self, *args, **kwargs): - EventMesh.publish("publish_AI_btnInfo",{"func":self.revoice_judger_cb}) - def revoice_judger_cb(self): - self.revoice_cnt += 1 - # print("语音留言回复定时器cnt = {}".format(self.revoice_cnt)) - if self.revoice_cnt > 20:#10S后退出播放,10S是为了给语音提示留时间 - self.revoice_cnt = 0 - self.voice_flag=0 - EventMesh.publish("set_1M_nowplay_flag",0) - print("语音留言回复停止") - self.revoice_timer.stop() - #音乐播放定时器 - def music_play(self, *args, **kwargs): - EventMesh.publish("publish_AI_btnInfo",{"func":self.music_play_cb}) - def music_play_cb(self): - self.music_cnt += 1 - if self.music_cnt > 4:#4S后播放音乐,4S是为了给语音提示留时间 - self.AI_VC_start() - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 - self.music_cnt = 0 - self.music_timer.stop() - #历史留言播放定时器 - def history_play(self, *args, **kwargs): - EventMesh.publish("publish_AI_btnInfo",{"func":self.history_play_cb}) - def history_play_cb(self): - self.his_cnt += 1 - if self.his_cnt > 4:#4S后播放,4S是为了给语音提示留时间 - self.AI_VC_start() - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 - self.his_cnt = 0 - self.his_timer.stop() - #未读留言是否播放定时器 - def wait_play(self, *args, **kwargs): - EventMesh.publish("publish_AI_btnInfo",{"func":self.wait_play_cb}) - def wait_play_cb(self): - self.wait_cnt += 1 - if self.wait_voicerupt_flag == 1:#按键被按下 - EventMesh.publish("set_is_listen_wait",1)#已听 - self.pwk_notice_flag = 1#表示进入收听未听留言 - self.AI_VC_start() - self.wait_sort = 1 - self.AI_TTS_str = "第"+str(self.wait_sort)+"条" - # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 - # self.wait_cnt = 0 - # print("进入留言定时器1，留言清空！！！！！") - # EventMesh.publish("update_play_new_voice_info")#更新已读状态 - self.wait_timer.stop() - if self.wait_cnt > 10:#10S后退出播放,5S是为了给语音提示留时间 - self.wait_cnt = 0 - self.recv_voice = 0#进入留言定时器，留言清空 - EventMesh.publish("set_history_voice_flag",0) - EventMesh.publish("set_wait_voice_flag",0) - EventMesh.publish("set_open_device",0)#开机不听 - self.wait_first_flag = 0 - if self.voice_choose_flag == 1:#播放完语音后，有冲突的时候关闭 - self.voice_choose_flag = 0 - print("进入留言定时器2，留言清空！！！！！") - self.wait_timer.stop() - #未读留言一条条播放定时器 - def wait_single_play(self, *args, **kwargs): - EventMesh.publish("publish_AI_btnInfo",{"func":self.wait_single_play_cb}) - def wait_single_play_cb(self): - self.wait_cnt += 1 - if self.wait_cnt > 2:#2S后播放,2S是为了给语音提示留时间 - self.AI_VC_start() - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 - self.wait_cnt = 0 - self.wait_timer.stop() - def set_voicerupt_flag(self,event,msg):#按键控制当前留言播放 - self.voicerupt_flag = msg - def set_wait_voicerupt_flag(self,event,msg):#按键控制未读留言播放 - self.wait_voicerupt_flag = msg - def get_voicerupt_flag(self,event,msg):#获取按键控制留言播放状态 - return self.voicerupt_flag - def get_recv_voice(self,event,msg): - return self.recv_voice - def set_recv_voice(self,event,msg): - return self.recv_voice - def put_record_data(self,data): - try: - if self._ptt_should_drop_inbound_tts(): - return - except Exception: - pass - #print("下发音频数据1:{}".format(len(data[0]))) - ret=self.AI_audio_queue.put(data) - try: - if self.tts_in_data_len == 0 and data and len(data[0]) > 0: - try: - sid = getattr(self, "_cur_sid", "") - did = getattr(self, "_cur_dialog_id", "") - except Exception: - sid, did = "", "" - print("[TTS] first_chunk len={} play_pcm={} VC_Play_flag={} sid={} did={}".format( - len(data[0]), - 1 if self.play_pcm != None else 0, - getattr(self, "VC_Play_flag", 0), - sid, - did, - )) - except Exception: - pass - self.tts_in_data_len += len(data[0]) - #print("下发音频数据2:{},{}".format(len(data[0]),self.tts_in_data_len)) - #msg[0]是tts播放字符串，msg[1]是是否关闭当前tts - def play_tts(self,event = None,msg= None): - EventMesh.publish("publish_AI_btnInfo",{"func":self.play_tts_cb,"data":msg}) - - def play_tts_cb(self,data): - try: - tts_str = data[0] - is_close_VC = data[1] - if tts_str != "": - self.AI_TTS_str = tts_str - if is_close_VC != None: - self.Close_flag = is_close_VC - if self.AI_VC_send_data_queue.empty()!=True: - self.AI_VC_send_data_queue.clear() - if self.AI_ASR_send_data_queue.empty()!=True: - self.AI_ASR_send_data_queue.clear() - try: - if is_close_VC: - EventMesh.publish("set_led_state", {"ai_state": "idle"}) - else: - EventMesh.publish("set_led_state", {"ai_state": "thinking"}) - except Exception: - pass - self.AI_ASR_send_buf_len = 0 - self._uplink_request_stop("play_tts_cb") - self.websocket_deconnect_VC() - self.AI_VC_start() - - # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - except Exception as e: - usys.print_exception(e) - pass - def check_text_server(self,cmd): - return -1 - - def _apply_vc_text_intent(self, text, cmd, llm_style_tts): - """表驱动文本意图（vc_text_intent）。未命中返回 None；命中返回 0/1/-1。""" - if vc_text_intent is None: - return None - try: - try: - _btv = bool("vc_config" in globals() and vc_config and vc_config.get_bool("BT_VOICE_FEATURES_ENABLED", False)) - except Exception: - _btv = False - r = vc_text_intent.match_vc_command(text, {"bt_voice": _btv}) - except Exception as e: - usys.print_exception(e) - return None - if not r: - return None - return self._apply_vc_text_intent_matched(r, cmd, llm_style_tts) - - def _apply_vc_text_intent_matched(self, r, cmd, llm_style_tts): - if r.get("needs_payload") and cmd is None: - return None - it = r["intent"] - if r.get("tts"): - self.AI_TTS_str = r["tts"] - if r.get("close_flag"): - self.Close_flag = 1 - if it == "goodbye": - return 0 - if it == "shutdown": - if llm_style_tts: - EventMesh.publish("publish_AI_btnInfo", {"func": self.Close}) - EventMesh.publish("Power_off") - return 1 - try: - close_only = ("vc_config" in globals() and vc_config and vc_config.get_bool("POWEROFF_CMD_CLOSE_ONLY", True)) - except Exception: - close_only = True - if close_only: - EventMesh.publish("publish_AI_btnInfo", {"func": self.Close}) - return 1 - EventMesh.publish("publish_AI_btnInfo", {"func": self.Close}) - EventMesh.publish("Power_off") - return 1 - if it == "toggle_dialog_mode": - if self.VC_mode == 0: - self.VC_mode = 1 - if llm_style_tts: - self.AI_TTS_str = "了解，已切换到单对话模式" - else: - try: - self.AI_TTS_str = vc_config.get_attr("CMD_TOGGLE_MODE_TO_PTT_TTS", "已切换到单对话模式") - except Exception: - self.AI_TTS_str = "已切换到单对话模式" - else: - self.VC_mode = 0 - if llm_style_tts: - self.AI_TTS_str = "了解，已切换到长对话模式" - else: - try: - self.AI_TTS_str = vc_config.get_attr("CMD_TOGGLE_MODE_TO_LONG_TTS", "已切换到长对话模式") - except Exception: - self.AI_TTS_str = "已切换到长对话模式" - EventMesh.publish("set_storage_config", ["VC_mode", self.VC_mode]) - return 0 - if it == "volume_set": - try: - pj = ujson.loads(cmd["command"]["params_json"]) - volume = pj["n"] - except Exception as e: - usys.print_exception(e) - try: - self.AI_TTS_str = cmd["command"]["fail_prompt"] - except Exception: - self.AI_TTS_str = "" - return -1 - if volume > 100 or volume < 0: - self.AI_TTS_str = "音量调整超出范围了，只能0-100哦" - else: - self.VC_TTS_volume = volume - self.aud.setVolume(self.VC_TTS_volume // 10) - EventMesh.publish("set_storage_config", ["VC_TTS_volume", self.VC_TTS_volume]) - self.AI_TTS_str = "了解，音量已经调整,当前音量是{}".format(self.VC_TTS_volume) - EventMesh.publish("post_device_state") - EventMesh.publish("adjust_volume_toBle", self.VC_TTS_volume) - return 0 - if it == "volume_up": - if llm_style_tts: - try: - print("self.VC_TTS_volume add:{}".format(self.VC_TTS_volume)) - except Exception: - pass - self.VC_TTS_volume += 10 - if self.VC_TTS_volume > 100: - self.AI_TTS_str = "音量已经是最大啦" if llm_style_tts else "音量已最大" - self.VC_TTS_volume = 100 - else: - self.aud.setVolume(self.VC_TTS_volume // 10) - EventMesh.publish("set_storage_config", ["VC_TTS_volume", self.VC_TTS_volume]) - EventMesh.publish("post_device_state") - EventMesh.publish("adjust_volume_toBle", self.VC_TTS_volume) - self.AI_TTS_str = ( - "了解，音量已经调大了,当前音量是{}".format(self.VC_TTS_volume) - if llm_style_tts - else "配置成功,当前音量 {}".format(self.VC_TTS_volume) - ) - return 0 - if it == "volume_down": - if llm_style_tts: - try: - print("self.VC_TTS_volume end:{}".format(self.VC_TTS_volume)) - except Exception: - pass - self.VC_TTS_volume -= 10 - if self.VC_TTS_volume < 0: - self.VC_TTS_volume = 0 - self.AI_TTS_str = "音量已经是最小啦" if llm_style_tts else "音量已最小" - else: - self.aud.setVolume(self.VC_TTS_volume // 10) - EventMesh.publish("set_storage_config", ["VC_TTS_volume", self.VC_TTS_volume]) - EventMesh.publish("post_device_state") - EventMesh.publish("adjust_volume_toBle", self.VC_TTS_volume) - self.AI_TTS_str = ( - "好嘞，音量已经调小了,当前音量是{}".format(self.VC_TTS_volume) - if llm_style_tts - else "配置成功,当前音量 {}".format(self.VC_TTS_volume) - ) - return 0 - if it in ("speech_faster", "speech_rate_up"): - self.speech_rate += 10 - if self.speech_rate > 50: - self.AI_TTS_str = "语速已经是最快了" - self.speech_rate = 50 - else: - EventMesh.publish("set_storage_config", ["speech_rate", self.speech_rate]) - self.AI_TTS_str = "明白，语速已经调快了,当前语速是{}".format(self.speech_rate) - return 0 - if it in ("speech_slower", "speech_rate_down"): - self.speech_rate -= 10 - if self.speech_rate < -50: - self.speech_rate = -50 - self.AI_TTS_str = "语速已经是最慢了" - else: - EventMesh.publish("set_storage_config", ["speech_rate", self.speech_rate]) - self.AI_TTS_str = "明白，语速已经调慢了,当前语速是{}".format(self.speech_rate) - return 0 - if it == "vad_faster": - self.vad_silence -= 200 - if self.vad_silence < 100: - self.AI_TTS_str = "已经是最快了" - self.vad_silence = 100 - else: - self.AI_TTS_str = "配置成功，当前语音断句速度是{}毫秒".format(self.vad_silence) - EventMesh.publish("set_storage_config", ["vad_silence", self.vad_silence]) - return 0 - if it == "vad_slower": - self.vad_silence += 200 - if self.vad_silence > 2000: - self.vad_silence = 2000 - self.AI_TTS_str = "已经是最慢了" - else: - self.AI_TTS_str = "配置成功，当前语音断句速度是{}毫秒".format(self.vad_silence) - EventMesh.publish("set_storage_config", ["vad_silence", self.vad_silence]) - return 0 - if it == "query_signal": - signal = net.csqQueryPoll() - self.AI_TTS_str = "当前信号{}".format(signal) - return 0 - if it == "query_battery": - battery = EventMesh.publish("get_battery") - try: - print("电量查询：{}".format(battery)) - except Exception: - pass - self.AI_TTS_str = "电量剩余百分之{}".format(battery) - return 0 - if it == "query_voltage": - Vbatt = Power.getVbatt() - self.AI_TTS_str = "当前电池电压{}".format(Vbatt) - return 0 - if it == "query_iccid": - iccid = sim.getIccid() - if iccid != -1 and len(iccid) > 5: - try: - print("iccid{}".format(iccid)) - except Exception: - pass - self.AI_TTS_str = "当前卡号为:{}".format(iccid) - else: - self.AI_TTS_str = "卡号查询失败" - return 0 - if it == "query_imei": - imei = modem.getDevImei() - if imei != -1 and len(imei) > 5: - try: - print("IMEI：{}".format(imei)) - except Exception: - pass - self.AI_TTS_str = "当前设备号为:{}".format(imei) - else: - self.AI_TTS_str = "设备号查询失败" - return 0 - if it == "query_version": - bt_versions = EventMesh.publish("get_bt_versions") - if bt_versions is None or bt_versions == "": - self.AI_TTS_str = "当前版本号:" + version_util.get_tts_version() - else: - self.AI_TTS_str = "当前版本号:" + version_util.get_tts_version(bt_versions) - return 0 - if it == "query_server": - self.AI_TTS_str = "服务器地址" + self.server_ip - return 0 - if it == "ota_check": - if self.ai_OTA.min_sys_ota() == 1: - self.AI_TTS_str = "有新版本，准备开始升级，升级过程会多次重启设备，请保证流量充足及电量充足" - else: - self.AI_TTS_str = "当前已是最新版本" - return 0 - if it == "factory_mode": - self.factory_mode() - return 1 - if it == "alarm_set": - try: - alarm = ujson.loads(cmd["command"]["params_json"]) - try: - print("params_json:{}".format(cmd["command"]["params_json"])) - except Exception: - pass - add_alarm_tmp = EventMesh.publish("AI_add_alarm", alarm["alarm"]) - if add_alarm_tmp[0] > 0: - self.AI_TTS_str = add_alarm_tmp[1] - return 1 - except Exception as e: - usys.print_exception(e) - try: - self.AI_TTS_str = cmd["command"]["fail_prompt"] - except Exception: - self.AI_TTS_str = "" - return -1 - if it == "alarm_clear": - try: - self.AI_TTS_str = EventMesh.publish("AI_delete_alarm") - except Exception as e: - usys.print_exception(e) - return 0 - if it == "alarm_list": - EventMesh.publish("read_alarm_list") - return 0 - if it == "voice_message_mode": - self.voice_flag = 1 - self.AI_TTS_str = "已进入语音留言模式，长按按钮开始录音，松开按钮结束" - self.Close_flag = 1 - self.stop_voice_timer.stop() - self.stop_voice_timer.start(30 * 1000, 0, self.exit_voice_message_timer) - try: - print("语音留言！！！！！！！！！！") - except Exception: - pass - return 0 - if it == "bt_ble_adv_on": - self.AI_TTS_str = "正在开蓝牙广播" - EventMesh.publish("AI_send_bt_ble_switch_toble", 1) - try: - print("正在开蓝牙广播") - except Exception: - pass - return 0 - if it == "bt_ble_adv_off": - self.AI_TTS_str = "正在关蓝牙广播" - EventMesh.publish("AI_send_bt_ble_switch_toble", 2) - try: - print("正在关蓝牙广播") - except Exception: - pass - return 0 - if it == "bt_classic_on": - self.AI_TTS_str = "正在开经典蓝牙" - EventMesh.publish("AI_send_bt_ble_switch_toble", 3) - try: - print("正在开经典蓝牙") - except Exception: - pass - return 0 - if it == "bt_classic_off": - self.AI_TTS_str = "正在关经典蓝牙" - EventMesh.publish("AI_send_bt_ble_switch_toble", 4) - try: - print("正在关经典蓝牙") - except Exception: - pass - return 0 - if it == "unbind_confirm": - self.AI_TTS_str = "设备解绑中...." - self.Close_flag = 1 - EventMesh.publish("send_unbind_dev_to_iot") - try: - print("解绑设备!!!!!!!!!!!!!!") - except Exception: - pass - return 0 - if it == "clear_earphone": - self.AI_TTS_str = "正在删除" - EventMesh.publish("clear_earphone_record") - try: - print("正在删除") - except Exception: - pass - return 0 - return None - - def check_text_llm(self,cmd): - try: - desc = cmd["command"]["chinese_description"] - if desc.find("切换音色") != -1: - text = cmd["command"]["params_json"] - try: - for key in self.VC_TTS_voice_type: - if text.find(key) != -1: - self.VC_TTS_voice_key = key - EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) - self.AI_TTS_str = "了解，声音已切换" - return 0 - except Exception as e: - usys.print_exception(e) - if text.find("男") != -1: - if "男" in self.VC_TTS_voice_type: - self.VC_TTS_voice_key = "男" - EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) - self.AI_TTS_str = "了解，声音已切换" - return 0 - if text.find("女") != -1: - if "女" in self.VC_TTS_voice_type: - self.VC_TTS_voice_key = "女" - EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) - self.AI_TTS_str = "了解，声音已切换" - return 0 - if text.find("同声") != -1 or text.find("ç«¥") != -1: - if "ç«¥" in self.VC_TTS_voice_type: - self.VC_TTS_voice_key = "ç«¥" - EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) - self.AI_TTS_str = "了解，声音已切换" - return 0 - if text.find("粤") != -1 or text.find("越野") != -1 or text.find("广东话") != -1: - if "粤" in self.VC_TTS_voice_type: - self.VC_TTS_voice_key = "粤" - EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) - self.AI_TTS_str = "了解，声音已切换" - return 0 - if text.find("四川") != -1: - if "四川" in self.VC_TTS_voice_type: - self.VC_TTS_voice_key = "四川" - EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) - self.AI_TTS_str = "了解，声音已切换" - return 0 - if text.find("个性") != -1: - if "个性" in self.VC_TTS_voice_type: - self.VC_TTS_voice_key = "个性" - EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) - self.AI_TTS_str = "了解，声音已切换" - return 0 - if text.find("英语") != -1: - if "英语" in self.VC_TTS_voice_type: - self.VC_TTS_voice_key = "英语" - EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) - self.AI_TTS_str = "了解，声音已切换" - return 0 - self.AI_TTS_str = "很抱歉，没找到对应声音" - return 0 - ret = self._apply_vc_text_intent(desc, cmd, True) - if ret is not None: - return ret - except Exception as e: - usys.print_exception(e) - try: - self.AI_TTS_str = cmd["command"]["fail_prompt"] - except Exception: - self.AI_TTS_str = "" - return -1 - - def check_text(self, text): - print("check_text:len:{}".format(len(text))) - if len(text) > 60: - return -1 - try: - if not isinstance(text, str): - return -1 - ret = self._apply_vc_text_intent(text, None, False) - if ret is not None: - return ret - except Exception as e: - usys.print_exception(e) - return -1 - - #获取trace_id - def get_trace_id(self): - t = utime.mktime(utime.localtime()) - t = t*1000+urandom.randint(0, 1000) - md5_data = "{}".format(t)#md5签名 - sign_data = "" - hash_obj = uhashlib.md5(md5_data) - res = hash_obj.digest() - - # 转成十六进制表示 - sign_data = ubinascii.hexlify(res) - sign_data = sign_data.decode('ascii') - return sign_data - - def get_session_id(self,data = "session_id"): - t = utime.mktime(utime.localtime()) - t = t*1000+urandom.randint(0, 1000) - md5_data = "{}{}".format(data,t)#md5签名 - sign_data = "" - hash_obj = uhashlib.md5(md5_data) - res = hash_obj.digest() - - # 转成十六进制表示 - sign_data = ubinascii.hexlify(res) - sign_data = sign_data.decode('ascii') - return sign_data - - def get_new_message_id(self): - message_id = self.get_trace_id()+"aaa"+self.get_trace_id() - self.VC_message_id=message_id[:32] - return self.VC_message_id - def get_new_task_id(self): - task_id = self.get_trace_id()+"aaa"+self.get_trace_id() - self.VC_task_id=task_id[:32] - return self.VC_task_id - #开始播放声音 - def VC_Play_Start(self): - try: - if self._ptt_should_skip_remote_playback(): - if self._vc_led_state_debug_enabled(): - print("[VC] VC_Play_Start skip (ptt_interrupt)") - return - except Exception: - pass - self.tts_in_data_len = 0 - self.tts_out_data_len = 0 - self._tts_first_write_ms = 0 - if self.VC_Play_flag == 0: - try: - _ptt_fast_open = False - try: - if getattr(self, "VC_mode", -1) == 1: - _ptt_fast_open = True - except Exception: - pass - # 本地提示音/文件播放可能占用同一音频通道，导致 TTS 流播放“写入成功但无声”； - # 在开启 TTS PCM 前先停掉本地播放，避免资源冲突。 - try: - self.aud.stopPlayStream() - self.aud.stopAll() - except Exception: - pass - # 资源互斥：播放前确保录音上行已停止（无 AEC 场景） - try: - if self.record_data_save_flag == 0 and self.VC_record_state_flag != 0: - self._uplink_request_stop("VC_Play_Start") - utime.sleep_ms(30) - except Exception: - pass - if self.VC_play_TYPE == VC_play_TYPE_PCM: - if self.play_pcm == None: - # 刚停采集时延后打开播音 PCM，且失败时重试，降低「pcm open write only fail」 - last_err = None - _open_retries = 1 if _ptt_fast_open else 3 - for i in range(_open_retries): - try: - if not _ptt_fast_open: - utime.sleep_ms(80 + i * 60) - self.play_pcm = audio.Audio.PCM(0, 1, self.VC_play_samplerate, 1, 1, 2) - ret = self.play_pcm.stream_write_start() - print("self.record_pcm.stream_write_start:ret:{}".format(ret)) - last_err = None - break - except Exception as e: - last_err = e - try: - if self.play_pcm != None: - self.play_pcm.close() - except Exception: - pass - self.play_pcm = None - if last_err: - raise last_err - elif self.VC_play_TYPE == VC_play_TYPE_MP3: - pass - self.VC_Play_flag = 1 - try: - if not self._ptt_should_skip_remote_playback(): - self._vc_publish_led("playing", "VC_Play_Start/tts") - except Exception: - pass - except Exception as e: - self.VC_Play_End() - usys.print_exception(e) - #暂停/继续声音播放 Pause_flag = 1暂停 - def VC_Play_Pause(self,Pause_flag): - try: - if self.VC_play_TYPE == VC_play_TYPE_PCM: - if self.play_pcm != None: - self.play_pcm.stream_pause(Pause_flag) - except Exception as e: - self.VC_Play_End() - usys.print_exception(e) - #停止播放声音 - def VC_Play_End(self): - self.VC_Play_flag = 0 - if self.VC_play_TYPE == VC_play_TYPE_PCM: - if self.VC_play_samplerate == 8000: - self.aud.stopPlayStream() - self.aud.stopAll() - elif self.VC_play_samplerate == 16000: - if self.play_pcm != None: - print("Close:play_pcm.close()!!!!!!!!!!!!!!!!!!") - try: - self.play_pcm.stream_stop() - self.play_pcm.close() - except Exception as e: - usys.print_exception(e) - self.play_pcm=None - - self.aud.stopPlayStream() - self.aud.stopAll() - elif self.VC_play_TYPE == VC_play_TYPE_MP3: - self.aud.stopPlayStream() - self.aud.stopAll() - if self.AI_audio_queue.empty()!=True: - self.AI_audio_queue.clear() - - # TTS 播完：按真实 FSM/录音态恢复灯；本地短音不走此路径 - try: - if getattr(self, "_vc_play_end_suppress_idle_led", 0): - if self._vc_led_state_debug_enabled(): - print("[LED] VC_Play_End skip led (suppress)") - elif getattr(self, "VC_record_state_flag", 0) != 0: - self._vc_publish_led("listening", "VC_Play_End/recording") - else: - self._vc_led_sync_from_fsm() - except Exception: - pass - # 关键日志：定位“tts 结束绿灯仍不灭”究竟是 base 态（如 charging/full）还是 playing 未退出 - try: - if "vc_config" in globals() and vc_config and ( - vc_config.get_bool("LED_DEBUG", False) or vc_config.get_bool("KEY_LED_DEBUG", False) or vc_config.get_bool("HW_STATUS_DEBUG", False) - ): - try: - led = EventMesh.publish("get_led_debug_state") - except Exception: - led = None - print("[VC] VC_Play_End led={}".format(led)) - except Exception: - pass - - - def _vc_tts_bytes_per_sec(self): - try: - return int(self.VC_play_samplerate) * 2 - except Exception: - return 32000 - - def _vc_try_play_end_tick(self): - """tts.completed 后判定本地 PCM 是否真正播完，避免过早 VC_Play_End 截断 TTS。""" - try: - if not getattr(self.fsm, "await_played", False): - return False - in_len = getattr(self, "tts_in_data_len", 0) - out_len = getattr(self, "tts_out_data_len", 0) - try: - q_empty = self.AI_audio_queue.empty() - except Exception: - q_empty = True - tts_done_ms = getattr(self.fsm, "last_tts_completed_ms", 0) - if not (in_len > 0 and out_len >= in_len and q_empty and tts_done_ms): - return False - - is_pcm_empty = (self.play_pcm is None) or ( - self.play_pcm is not None and self.play_pcm.stream_is_empty() == 1 - ) - pcm_buf = 0 - if self.play_pcm is not None: - try: - pcm_buf = self.play_pcm.stream_get_buf_len() - except Exception: - pcm_buf = -1 - - now = utime.ticks_ms() - bps = self._vc_tts_bytes_per_sec() - total_ms = int(out_len * 1000 // bps) if bps > 0 else 0 - fw = getattr(self, "_tts_first_write_ms", 0) or tts_done_ms - try: - elapsed_at_done = utime.ticks_diff(tts_done_ms, fw) - except Exception: - elapsed_at_done = 0 - if elapsed_at_done < 0: - elapsed_at_done = 0 - remain_ms = max(400, total_ms - elapsed_at_done) if total_ms > 0 else 400 - try: - since_done = utime.ticks_diff(now, tts_done_ms) - except Exception: - since_done = 0 - tail_ok = since_done >= remain_ms - pcm_ok = is_pcm_empty and (pcm_buf == 0 or pcm_buf == -1) - too_long = since_done > max(remain_ms + 2000, 12000) - - ready = (pcm_ok and tail_ok) or too_long - if not ready: - try: - if not getattr(self, "_tts_play_check_log_ms", 0) or utime.ticks_diff( - now, self._tts_play_check_log_ms - ) >= 800: - self._tts_play_check_log_ms = now - print( - "[TTS] play_end_wait since_done={}ms remain~{}ms total~{}ms " - "in={} out={} q_empty={} pcm_buf={} pcm_empty={}".format( - since_done, remain_ms, total_ms, in_len, out_len, - 1 if q_empty else 0, pcm_buf, 1 if is_pcm_empty else 0, - ) - ) - except Exception: - pass - return False - - try: - print( - "[TTS] play_end_fire since_done={}ms remain~{}ms total~{}ms " - "in={} out={} pcm_buf={} too_long={}".format( - since_done, remain_ms, total_ms, in_len, out_len, pcm_buf, 1 if too_long else 0, - ) - ) - except Exception: - pass - EventMesh.publish("publish_AI_btnInfo", {"func": self.VC_Play_End}) - self._fsm_apply_actions(self.fsm.on_local_play_end()) - try: - self.fsm.reset_rx_phases() - except Exception: - pass - self._tts_first_write_ms = 0 - return True - except Exception as e: - usys.print_exception(e) - return False - - def _1S_timer_cb(self,*argc): - EventMesh.publish("publish_AI_btnInfo",{"func":self._1S_timer_cb_}) - - def _fsm_timeout_table_tick(self, now_ms): - # 兜底统一入口：按“状态机状态 + 超时阈值 + 节流”驱动，避免兜底抢跑/多入口打架。 - # 只在对应 turn/session 状态触发动作。 - try: - turn = getattr(self.fsm, "turn", VCTurnState.Idle) - except Exception: - turn = VCTurnState.Idle - - # 1) synthesis→tts.started 超时：只在 Thinking 期允许清 pending 并重启会话 - # 根因：服务端未返回 tts.started 时，继续“回到聆听/开麦”会把本轮状态机拉扯得更乱； - # 这里直接重启会话，让协议重新进入 started→下一轮流程。 - try: - if turn == VCTurnState.Thinking and self._tts_pending == 1 and self._tts_pending_ms: - try: - if self._fb_last_tts_pending_timeout_ms: - tts_gap_ok = utime.ticks_diff(now_ms, self._fb_last_tts_pending_timeout_ms) > 15000 - else: - tts_gap_ok = True - except Exception: - tts_gap_ok = True - if utime.ticks_diff(now_ms, self._tts_pending_ms) > 8000 and tts_gap_ok: - self._fb_last_tts_pending_timeout_ms = now_ms - print("[VC][tts_pending] timeout, clear pending and restart session") - self._tts_pending = 0 - self._tts_pending_ms = 0 - if self.VC_mode == 0 and self.Close_flag == 0: - try: - self.fsm.reset_rx_phases() - except Exception: - pass - EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_VC_start}) - except Exception: - pass - - # 2) thinking 超时：只在「llm.completed 已收到但 tts.started 长时间未到」时才重启会话 - # 注意：不能以 llm.created 的时间戳作为超时基准，否则会在 LLM 正常推理期间误判超时（notts 场景）。 - try: - if self.VC_mode == 0 and self.Close_flag == 0: - llm_completed_ms = getattr(self, "_llm_completed_ms", 0) - # 只有在 llm.completed 之后才允许超时重启 - if turn == VCTurnState.Thinking and llm_completed_ms and self._turn_has_tts_started == 0: - age_ms = utime.ticks_diff(now_ms, llm_completed_ms) - restart_threshold_ms = 20000 - # _fb_last_thinking_restart_ms==0 时 ticks_diff(now,0) 非预期（间隔被误判极大）， - # 会在 llm.completed 后首批 tick 就触发 AI_VC_start → notts（无 tts.started）；首次仅信任 age 门限 - try: - if self._fb_last_thinking_restart_ms: - gap_ok = utime.ticks_diff(now_ms, self._fb_last_thinking_restart_ms) > 15000 - else: - gap_ok = True - except Exception: - gap_ok = True - if (age_ms > restart_threshold_ms and gap_ok and - self._tts_pending == 0 and self._need_audio_played_ack == 0 and self.VC_Play_flag == 0): - self._fb_last_thinking_restart_ms = now_ms - self._llm_completed_ms = 0 - self._server_vad_silence = 0 - try: - self.fsm.reset_rx_phases() - except Exception: - pass - try: - print("[VC][thinking_timeout] AI_VC_start age_ms={} thr={}".format( - age_ms, restart_threshold_ms)) - except Exception: - pass - self._vc_led_alert_rb_flash() - # thinking_timeout 策略：重启会话（而不是抢开麦），避免干扰服务端 LLM/TTS 流程 - EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_VC_start}) - except Exception: - pass - - def _1S_timer_cb_(self): - try: - self._vc_tick_serial_cancel_timeout() - except Exception: - pass - - EventMesh.publish("system_sleep_rest")#防止黑屏 - # --- Level 1：仅长对话(VC_mode==0)，无业务下行 idle_sec 后结束会话（需再次按键进入） - # close_timerout：保活清除策略见 _vc_rx_resets_long_idle_timer（排除文本 ping/pong、heartbeat 等） - # 与浅休眠/灭屏解耦；单对话不走本计时 - idle_sec = 20 - try: - if "vc_config" in globals() and vc_config: - idle_sec = int(vc_config.get_int("VC_IDLE_AUTO_CLOSE_SEC", 20)) - except Exception: - idle_sec = 20 - if idle_sec < 5: - idle_sec = 5 - if self.VC_mode == 0 and self.VC_satrt_flag == 1 and getattr(self, "AI_client_VC_isconnect", 0) == 1: - self.close_timerout += 1 - else: - self.close_timerout = 0 - if self.VC_mode == 0 and idle_sec > 0 and self.close_timerout > idle_sec: - try: - print("[VC][idle_L1] 长对话无业务下行超时: elapsed={}s thr={}s VC_mode={} VC_satrt={} ws_conn={} {}".format( - self.close_timerout, idle_sec, self.VC_mode, - getattr(self, "VC_satrt_flag", -1), getattr(self, "AI_client_VC_isconnect", -1), - self._vc_fsm_log_snip())) - except Exception: - pass - if self._vc_long_idle_active_speech_or_later(): - try: - print("[VC][idle_L1] 处理=websocket_error（对话中无服务端响应）") - except Exception: - pass - print("[VC][long_idle] branch=speech_no_rsp -> websocket_error + ud_error") - print("接口无响应，超时退出") - EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_error,"data":"接口超时了。请重试"}) - self.ud_error(["VC",self.VC_taskid]) - else: - try: - print("[VC][idle_L1] 处理=Close（结束对话，需再次按键进入）") - except Exception: - pass - print("[VC][long_idle] branch=idle_timeout -> Close") - EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) - #EventMesh.publish("sleep_BT")#休眠蓝牙 - - # --- Level 2：长对话+单对话，业务 idle 累计 VC_IDLE_DEEP_SLEEP_SEC → 断开（若仍连）+ 深睡事件 - deep_sec = 60 - deep_en = 1 - try: - if "vc_config" in globals() and vc_config: - deep_sec = int(vc_config.get_int("VC_IDLE_DEEP_SLEEP_SEC", 60)) - deep_en = int(vc_config.get_int("VC_IDLE_DEEP_SLEEP_ENABLE", 1)) - except Exception: - deep_sec, deep_en = 60, 1 - active_deep = bool(deep_en) and deep_sec > 0 - if active_deep and deep_sec < 10: - deep_sec = 10 - if not active_deep: - self._idle_deep_sec = 0 - elif not self._vc_is_business_idle(): - self._idle_deep_sec = 0 - try: - self._vc_idle_deep_armed_suppress = 0 - except Exception: - pass - elif getattr(self, "_vc_idle_deep_armed_suppress", 0): - pass - else: - self._idle_deep_sec += 1 - if active_deep and self._idle_deep_sec > deep_sec: - try: - print("[VC][idle_L2] 业务idle累计超时: elapsed={}s thr={}s VC_mode={} VC_satrt={} ws_conn={}".format( - self._idle_deep_sec, deep_sec, self.VC_mode, - getattr(self, "VC_satrt_flag", -1), getattr(self, "AI_client_VC_isconnect", -1))) - except Exception: - pass - try: - self._vc_idle_deep_armed_suppress = 1 - except Exception: - pass - self._idle_deep_sec = 0 - EventMesh.publish("publish_AI_btnInfo", {"func": self._vc_idle_deep_sleep_impl}) - - # 统一兜底表（按状态机状态触发） - try: - self._fsm_timeout_table_tick(utime.ticks_ms()) - except Exception: - pass - - # （不做额外兜底）动态切换依赖 FSM 状态正确清理后，由 server.tts.completed→local.play_end 驱动自动开麦 - - # print("get_audio_state !!!!!{} {} {}\r\n".format(self.play_end,EventMesh.publish("get_audio_state"),self.play_flag)) - # ===== 播放结束检测：用 FSM 驱动 play_end 动作链 ===== - try: - self._vc_try_play_end_tick() - except Exception: - pass - if self.online_kws == 1: - if self.VC_satrt_kws_flag == 1 and self.VC_satrt_flag == 0:#要打开录音上传数据实现kws - pass - - # thinking/llm→tts 等兜底已在 _fsm_timeout_table_tick 内统一处理 - # VC WebSocket 文本 ping：间隔由 default_config 的 VC_WS_PING_INTERVAL_SEC 控制（秒），<=0 表示关闭 ping - try: - _ping_iv = int(vc_config.get_int("VC_WS_PING_INTERVAL_SEC", 4)) - except Exception: - _ping_iv = 4 - if _ping_iv > 0: - self.VC_pingping_timerout += 1 - if self.VC_pingping_timerout >= _ping_iv: - self.VC_pingping_timerout = 0 - self.AI_client_VC_ping() - # self.asr_pingping_timerout += 1 - # if self.asr_pingping_timerout>3: - # self.asr_pingping_timerout=0 - # self.AI_client_asr_ping() - # 历史遗留：VC_Thinking_timerout 的重启通道会在 llm.created 后抢跑触发 AI_VC_start， - # 直接打断本轮 LLM→TTS（notts 日志：llm.created 后立刻 AI_VC_start，导致无 tts.started）。 - # 现在统一由 _fsm_timeout_table_tick(Thinking 超时) 负责重启会话，因此禁用该旧通道。 - if self.VC_Thinking_timerout > 0 and self.VC_Thinking_timerout < 100: - self.VC_Thinking_timerout = 100 - # def AI_kws_start(self): - # print("AI_kws_start") - # EventMesh.publish("wakeup_BT") - # EventMesh.publish("system_sleep_rest")#防止黑屏 - # url="ws://118.145.158.207:22080/wakeup/ws" - # self.AI_client_kws = uwebsocket.Client.connect(url, debug=True) - # self.AI_client_VC_recv_id = _thread.start_new_thread(self.AI_client_VC_recv, (self.AI_client_VC,)) - # self.VC_Speech_Start()#开启线程获取录音数据 - #VC开始 - def AI_VC_start(self): - print("AI_VC_start1") - - EventMesh.publish(constants.EVENT.SYSTEM_UP)#唤醒休眠 - self.VC_satrt_flag = 1 - self.VC_stop_play_audio_flag = 1 - self.VC_satrt_kws_flag = 1 - if self.VC_Thinking_timerout < 100: - self.VC_Thinking_timerout = 0 - # 不在这里提前切灯态：播放/聆听灯只在“播放/录音真正开始/停止”处更新 - EventMesh.publish("wakeup_BT") - #self.VC_Play_End()#关闭音频播放 - print("AI_VC_start2") - #self.aud.aud_tone_play(16,200)#播放按键音 - self.LocalRespondingEnded_timerout = 0 - self.close_timerout = 0 - self._idle_deep_sec = 0 - self._vc_idle_deep_armed_suppress = 0 - - # - if self.AI_client_VC_isconnect == 0:#没连接的时候请求开始 - self.websocket_connect_VC()#连接服务器 - - #utime.sleep_ms(50) - else: - print("AI_VC_start2 (already connected) {}".format(self._vc_fsm_log_snip())) - EventMesh.publish(constants.EVENT.SYSTEM_UP)#唤醒休眠 - if self.AI_client_VC_isconnect == 0: - #self.Close() - return - # 单对话 PTT：会话已 started 时勿重复发 Start（否则第二轮起上行/ASR 易乱） - _ptt_skip_start = False - try: - if self.VC_mode == 1 and getattr(self, "_vc_session_started", 0) == 1: - _ptt_skip_start = True - print("[PTT] AI_VC_start skip duplicate Start (session active)") - except Exception: - pass - if not _ptt_skip_start: - self.AI_client_VC_start_requset() - print("AI_VC_start3") - - if (not _ptt_skip_start) and self.AI_TTS_str == "" and self.AI_new_voice_url == "": - self._uplink_request_start("AI_VC_start") - - - # pass - print("AI_VC_start4") - - #self.VC_Play_Start() - #EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Play_Start}) - #开始线程 - - # if self.AI_VC_send_data_queue_thread_id == 0: - # _thread.stack_size(self.Audio_buf_size+8*1024) - # self.AI_VC_send_data_queue_thread_id = _thread.start_new_thread(self.AI_VC_send_data_queue_thread, ()) - # _thread.stack_size(8*1024) - # print("AI_VC_start :AI_VC_send_data_queue_thread_id :{} ".format(self.AI_VC_send_data_queue_thread_id)) - - if self.AI_audio_queue_thread_id ==0: - _thread.stack_size(24*1024) - self.AI_audio_queue_thread_id = _thread.start_new_thread(self.AI_audio_queue_thread, ()) - _thread.stack_size(8*1024) - print("AI_VC_start :AI_audio_queue_thread_id :{} ".format(self.AI_audio_queue_thread_id)) - else: - print("AI_VC_start :AI_audio_queue_thread_id != 0 ") - EventMesh.publish(constants.EVENT.SYSTEM_UP)#唤醒休眠 - self._1S_timer.start(1000,1,self._1S_timer_cb) - # self.VC_pingping_timerout = 0 - # self.asr_pingping_timerout = 0 - print("AI_VC_start5") - #开始会话 - def AI_client_VC_start_requset(self): - voice="zh_male_wennuanahu_moon_bigtts" - try: - voice = self.VC_TTS_voice_type[self.VC_TTS_voice_key]["voice"] - except Exception as e: - usys.print_exception(e) - #voice="zh_female_dongmanxiaoyang_moon_bigtts" - token = EventMesh.publish("get_dev_token") - if token ==None: - token = "" - # bot_option：优先从 default_config（usr 可覆盖）读取 VC_BOT_ID / VC_WORKFLOW_ID，允许空字符串 - _def_bot_id = "7462617881076351002" - _def_workflow_id = "7462621757327196187" - try: - if "vc_config" in globals() and vc_config: - _vc_bot_id = vc_config.get_attr("VC_BOT_ID", _def_bot_id) - _vc_workflow_id = vc_config.get_attr("VC_WORKFLOW_ID", _def_workflow_id) - if _vc_bot_id is None: - _vc_bot_id = "" - if _vc_workflow_id is None: - _vc_workflow_id = "" - else: - _vc_bot_id, _vc_workflow_id = _def_bot_id, _def_workflow_id - except Exception: - _vc_bot_id, _vc_workflow_id = _def_bot_id, _def_workflow_id - json_data = \ - { - "header": { - "event": "Start", - "trace_id": self.get_trace_id(), - }, - "payload": { - "simple_response": True,#不返回llm过程文本 - "device_id": modem.getDevImei(), - - "user_agent": "AudioClient/1.0", - "dialog_attributes": { - #"provider": "doubao", - # "provider": "coze", - - "cfg": { - "disable_empty_prompt" : True,#拦截空字符请求大模型 - "custom_variables": { - "name": self.bot_name, - "devToken":token, - }, - "greeting":{ - "enabled":False, - "content":"" - }, - "openai_option":{ - }, - - "bot_option": { - "use_workflow": True, - #"input_format": "text", - "bot_id": _vc_bot_id, - "workflow_id": _vc_workflow_id, - - } - } - }, - "outbound_attributes": { - "provider": "volcengine", - "cfg": { - "format": "pcm", - "sample_rate": self.VC_play_samplerate, - "bit_rate": 16, - "channel": 1, - "volume": self.VC_TTS_volume, - "emotion": self.emotion, - #"voice": "7426720361732980745", - "voice": voice,#音色 - "speech_rate": self.speech_rate,#语速 - "chunk_interval_ms": 10, # 配置每段音频入队间隔时间 100ms - } - }, - "inbound_attributes": { - # "provider": "xun-fei",# - "provider": "volcengine", - "disable_vad":True, - "vad_silence":self.vad_silence, - "interrupt_options":{ - "concatenate_results":True, - "thinking_time_only": True, - #"keyword_interrupt": false, // 启用打断词功能 （打断词只有在思考以及之后才能打断） - #"interrupt_keywords": ["停止", "打断","打断一下", "等一下", "别说了", "打住"] // 配置打断词列表 - }, - "cfg": { - "format": "pcm", - "sample_rate": self.VC_record_samplerate, - "bit_rate": 16, - "channel": 1, - "enable_punctuation": True, - "enable_itn":True, - "keywords":[] - } - }, - "commands":self.commands, - } - } - # "bot_id":"7481164663863459850",#鸭子 - # "workflow_id":"7480485093845991443", - # "bot_id":"7462617881076351002",#语言助手 - # "workflow_id":"7462621757327196187", - # "bot_id":"7469964258592931852",#深圳百事通 - # "workflow_id":"7469968083063603241", - # "bot_id":"7411071327182438412",#柒牌 - # "workflow_id":"7469049680291610636", - # if self.online_kws == 1:#打开 - # json_data["payload"]["inbound_attributes"]["interrupt_options"]["keyword_interrupt"] = True - # json_data["payload"]["inbound_attributes"]["interrupt_options"]["interrupt_keywords"] = ["你好，柒先生","你好"] - #如果 - json_data["payload"]["outbound_attributes"]["provider"]=self.outbound_attributess_provider#TTS音色服务选择，火山 - json_data["payload"]["inbound_attributes"]["provider"]=self.inbound_attributess_provider#ASR服务选择，火山 - #优先选择self.VC_TTS_voice_type[self.VC_TTS_voice_key]里得参数 - # if "inbound_attributess_provider" in self.VC_TTS_voice_type[self.VC_TTS_voice_key]: - # if len(self.VC_TTS_voice_type[self.VC_TTS_voice_key]["inbound_attributess_provider"])>0: - # json_data["payload"]["inbound_attributes"]["provider"]=self.VC_TTS_voice_type[self.VC_TTS_voice_key]["inbound_attributess_provider"] - # print("inbound_attributess_provider是VC_TTS_voice_type里的：{}".format(self.VC_TTS_voice_type[self.VC_TTS_voice_key]["inbound_attributess_provider"])) - if "outbound_attributess_provider" in self.VC_TTS_voice_type[self.VC_TTS_voice_key]: - if len(self.VC_TTS_voice_type[self.VC_TTS_voice_key]["outbound_attributess_provider"])>0: - json_data["payload"]["outbound_attributes"]["provider"]=self.VC_TTS_voice_type[self.VC_TTS_voice_key]["outbound_attributess_provider"] - print("outbound_attributess_provider是VC_TTS_voice_type里的：{}".format(self.VC_TTS_voice_type[self.VC_TTS_voice_key]["outbound_attributess_provider"])) - - # 根据配置设置默认 bot/workflow，并允许 VC_TTS_voice_type 覆盖 - try: - if "vc_config" in globals() and vc_config: - opt = vc_config.get_attr("DIALOG_BOT_OPTION", None) - if isinstance(opt, dict): - if "bot_id" in opt: - json_data["payload"]["dialog_attributes"]["cfg"]["bot_option"]["bot_id"] = opt.get("bot_id") or "" - if "workflow_id" in opt: - json_data["payload"]["dialog_attributes"]["cfg"]["bot_option"]["workflow_id"] = opt.get("workflow_id") or "" - if "use_workflow" in opt: - json_data["payload"]["dialog_attributes"]["cfg"]["bot_option"]["use_workflow"] = bool(opt.get("use_workflow")) - except Exception: - pass - - try: - # voice 配置可覆盖 bot_id/workflow_id/openai_option - cur = self.VC_TTS_voice_type[self.VC_TTS_voice_key] - if isinstance(cur, dict): - if cur.get("bot_id"): - json_data["payload"]["dialog_attributes"]["cfg"]["bot_option"]["bot_id"] = cur.get("bot_id") - if "workflow_id" in cur: - json_data["payload"]["dialog_attributes"]["cfg"]["bot_option"]["workflow_id"] = cur.get("workflow_id") or "" - if cur.get("openai_option"): - json_data["payload"]["dialog_attributes"]["cfg"]["openai_option"] = cur.get("openai_option") - except Exception: - pass - - try: - _bo = json_data["payload"]["dialog_attributes"]["cfg"]["bot_option"] - if not _bo.get("bot_id", "") or not _bo.get("workflow_id", ""): - _bo["use_workflow"] = False - except Exception: - pass - try: - if "vc_config" in globals() and vc_config: - p = vc_config.get_attr("DIALOG_PROVIDER_DEFAULT", None) - if p: - json_data["payload"]["dialog_attributes"]["provider"] = p - except Exception: - pass - - try: - if "vc_config" in globals() and vc_config and vc_config.get_bool("KEYWORDS_FROM_VOICE_TYPE", False): - keywords = [] - for key in self.VC_TTS_voice_type: - keywords.append(key) - json_data["payload"]["inbound_attributes"]["cfg"]["keywords"] = keywords - except Exception: - pass - if self.dialog_attributes_provider != None: - json_data["payload"]["dialog_attributes"]["provider"] = self.dialog_attributes_provider - if self.commands_mode !=0 : - json_data["payload"]["commands"] = [] - if self.VC_mode == 0 :#and dev_type != "7PIN_BK_coze":#长对话模式打开vad - json_data["payload"]["inbound_attributes"]["disable_vad"] = False - # 新会话开始：在收到服务端 started 前，禁止发送音频帧（避免 session not initialized） - self._vc_session_started = 0 - try: - self.fsm.on_session_start_request() - except Exception: - pass - self.AI_client_VC_send(ujson.dumps(json_data)) - try: - _tid = json_data.get("header", {}).get("trace_id", "") - print("[VC][proto] Start enqueued trace_id={} VC_mode={} disable_vad={} ws_conn={}".format( - _tid or "-", - self.VC_mode, - json_data.get("payload", {}).get("inbound_attributes", {}).get("disable_vad", "?"), - self.AI_client_VC_isconnect)) - except Exception: - pass - - #开始上传语音 - def AI_client_VC_send_speech(self): - json_data = \ - { - "header": { - "event": "audio.create", - - }, - } - - if self.VC_mode == 1:#单对话模式 - # 协议门控（可用 VC_PROTO_GUARDS 临时关闭） - if (not VC_PROTO_GUARDS) or (self._vc_session_started == 1): - self._ptt_audio_created = 1 - try: - print("[VC][proto] audio.create send session_started={} proto_guards={}".format( - self._vc_session_started, VC_PROTO_GUARDS)) - except Exception: - pass - self.AI_client_VC_send(ujson.dumps(json_data)) - else: - try: - print("[VC][proto] audio.create SKIP session_started={} VC_PROTO_GUARDS={} " - "(等对端 started 后再开麦)".format(self._vc_session_started, VC_PROTO_GUARDS)) - except Exception: - pass - - - - - - - try: - self.fsm.on_server_asr_started() - except Exception: - pass - #结束上传语音 - def AI_client_VC_end_speech(self): - json_data = \ - { - "header": { - "event": "audio.completed", - - }, - } - self.AI_client_VC_send(ujson.dumps(json_data)) - self._ptt_audio_created = 0 - EventMesh.publish("publish_AI_btnInfo",{"func":self._uplink_request_stop, "data":"audio.completed"}) - - - #客户端打断对话 - def AI_client_VC_RequestToSpeak(self): - json_data = \ - { - "event": "cancel", - "payload": {} - } - print("AI_client_VC_RequestToSpeak!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") - # 模式切换等场景 play_tts(..., is_close_VC=1) 会置 Close_flag=1；用户再按 AI 键走 cancel→canceled→ - # uplink_start 时，FSM 与 _uplink_request_start 均会因 Close_flag 拒绝开麦，表现为“切回长对话后按键无上行”。 - try: - if self.VC_mode == 0: - self.Close_flag = 0 - self._vc_pending_interrupt_uplink = 1 - self._vc_clear_stale_record_state("RequestToSpeak") - except Exception: - pass - # 只做“打断+清播放队列”，不要在这里关音频队列线程（会导致后续 TTS/提示音不稳定，且触发竞态） - EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_init}) - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_send,"data":ujson.dumps(json_data)}) - utime.sleep_ms(1) - try: - self.fsm.reset_rx_phases() - except Exception: - pass - # #self.Close() - # self.AI_VC_start() - #TTS播放 - def AI_client_VC_synthesis(self): - # 幂等保护：在上一条 synthesis 还未等到 tts.started 前，不重复发（防止多兜底/多入口重复触发） - try: - if self._tts_pending == 1: - return - except Exception: - pass - json_data = \ - { - "event": "synthesis", - "payload": { - "text": self.AI_TTS_str - } - } - # 语义标记只用于“是否自动开麦”的策略。 - # 当前策略：长对话不自动开麦（需按键开始），因此这里不对提示音做自动开麦标记。 - self.AI_TTS_str = "" - # 标记为“等待 tts.started”，避免在 synthesis→tts.started 窗口误开下一轮录音上行 - self._tts_pending = 1 - try: - self._tts_pending_ms = utime.ticks_ms() - except Exception: - self._tts_pending_ms = 0 - try: - self.fsm.on_local_synthesis_sent() - except Exception: - pass - print("AI_client_VC_synthesis!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") - # if self.VC_state == VC_State_RequestToRespond: - # print("AI_client_VC_RequestToRespond 重复 tts_str:{}\nasr_str:{}".format(self.AI_TTS_str,self.AI_ASR_str)) - self.AI_client_VC_send(ujson.dumps(json_data)) - # 金三 V2.5：AudioPlayed — 端侧 TTS 播报结束后上报，服务端进入下一轮对话（轻度重同步，不断开 WebSocket） - def AI_client_VC_audio_played(self): - # V2.5：播完上报；与 VC_PROTO_GUARDS 无关（见文件头说明） - try: - if self.AI_client_VC_isconnect != 1 or self.AI_client_VC == None: - return - json_data = {"event": "audio.played", "payload": {}} - print("AI_client_VC_audio_played") - self.AI_client_VC_send(ujson.dumps(json_data)) - except Exception as e: - usys.print_exception(e) - def kws_even(self,event = None,msg= None): - EventMesh.publish("publish_AI_btnInfo",{"func":self.kws_even_cb}) - def kws_even_cb(self): - if EventMesh.publish("get_sos_flag") == 1: - return - print("kws_even_cb") - if self.VC_mode == 1: - try: - tts_str = vc_config.get_attr("KWS_PTT_HINT", "请您按住圆形的按钮开始讲话，讲完松开即可 ") - except Exception: - tts_str = "请您按住圆形的按钮开始讲话，讲完松开即可 " - EventMesh.publish("play_tts",[tts_str,1]) - return - if self.VC_mode == 0: - if EventMesh.publish("get_voice_record_flag") == 1:#停止静默录音 - if EventMesh.publish("get_voice_record_mode") == 2:#语音留言不能被打断 - return - elif EventMesh.publish("get_voice_record_mode") == 1: #静默录音需要打断 - EventMesh.publish("stop_voice_record")#停止静默录音 - self.aud.aud_tone_play(10,400) - # EventMesh.publish("audio_montage_play",["chenggong.mp3"]) - self.Close() - # if dev_type == "7PIN" : - # self.aud.aud_tone_play(10,400)#播放按键音 - #self.aud.aud_tone_play(10,200)#播放按键音 - self.AI_TTS_str = ""#"你好，有什么吩咐" - self.AI_ASR_str = "" - self.AI_ASR_str_tmp = "" - self._uplink_request_stop("kws_even_cb") - self._uplink_request_start("kws_even_cb") - self.AI_VC_start() - - def set_vad_silence(self,event = None,msg= None): - self.vad_silence = msg - EventMesh.publish("set_storage_config",["vad_silence",self.vad_silence]) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.play_new_voice_cb,"data":msg}) - #设置tts参数 - def set_wait_first_flag(self,event = None,msg= None):#设置未读语音播报初次记录flag - self.wait_first_flag = msg - def set_VC_TTS_voice_type(self,event = None,msg= None): - self.VC_TTS_voice_type["APP_voice"] = msg - #self.VC_TTS_voice_type[self.VC_TTS_voice_key] - self.VC_TTS_voice_key = "APP_voice" - EventMesh.publish("set_storage_config",["VC_TTS_voice_type",self.VC_TTS_voice_type]) - EventMesh.publish("set_storage_config",["VC_TTS_voice_key",self.VC_TTS_voice_key]) - print("[VC] TTS_voice key={} map={}".format(self.VC_TTS_voice_key, self.VC_TTS_voice_type)) - #修改音量大小 msg为1则为加大，msg为0则为减小,返回1成功，返回0失败 - def change_volume(self,event =None, msg =None): - if msg == 0 : - self.VC_TTS_volume -= 10 - if self.VC_TTS_volume < 0: - self.VC_TTS_volume = 0 - return 0 - else: - self.aud.setVolume(self.VC_TTS_volume //10) - EventMesh.publish("set_storage_config",["VC_TTS_volume",self.VC_TTS_volume]) - EventMesh.publish("post_device_state") - EventMesh.publish("adjust_volume_toBle",self.VC_TTS_volume) - return 1 - elif msg == 1: - self.VC_TTS_volume += 10 - if self.VC_TTS_volume > 100: - self.VC_TTS_volume = 100 - return 0 - else: - self.aud.setVolume(self.VC_TTS_volume //10) - EventMesh.publish("set_storage_config",["VC_TTS_volume",self.VC_TTS_volume]) - EventMesh.publish("post_device_state") - EventMesh.publish("adjust_volume_toBle",self.VC_TTS_volume) - return 1 - elif msg >= 100:#修改为具体音量 - if msg > 200: - self.VC_TTS_volume = 100 - return 0 - else: - self.VC_TTS_volume = msg - 100#还原数据 - self.aud.setVolume(self.VC_TTS_volume //10) - EventMesh.publish("set_storage_config",["VC_TTS_volume",self.VC_TTS_volume]) - EventMesh.publish("post_device_state") - EventMesh.publish("adjust_volume_toBle",self.VC_TTS_volume) - return 1 - #调节聆听快慢（以秒为单位） - def NLP_set_vad_silence(self,event,msg): - if msg == 1:#聆听变快 - self.vad_silence -= 1000 - if self.vad_silence < 1000: - AI_TTS_str = "已经是最快了" - self.vad_silence = 1000 - else: - AI_TTS_str = "配置成功，当前语音断句速度是{}秒".format(self.vad_silence/1000) - elif msg == 0:#聆听变慢 - self.vad_silence += 1000 - if self.vad_silence > 4000: - self.vad_silence = 4000 - #EventMesh.publish("audio_montage_play",["yinliangyizuixiao.mp3"]) - #self.aud.play(1,0,'U:/static/yinliangyizuixiao.mp3')#音量已最小 - AI_TTS_str = "已经是最慢了" - else: - AI_TTS_str = "配置成功，当前语音断句速度是{}秒".format(self.vad_silence/1000) - EventMesh.publish("set_storage_config",["vad_silence",self.vad_silence]) - return AI_TTS_str - #查询指令判断 - def find_cmd(self,event ,msg): - if "电量" in msg: - battery = EventMesh.publish("get_battery") - print("电量查询：{}".format(battery)) - tts_play_str = "电量剩余百分之{}".format(battery) - EventMesh.publish("play_tts",[tts_play_str,0]) - # print("电量！！！！！！！！！！！") - elif "信号" in msg: - signal = net.csqQueryPoll() - tts_play_str = "当前信号{}".format(signal) - EventMesh.publish("play_tts",[tts_play_str,0]) - # print("信号！！！！！！！！！！！") - elif "卡号" in msg: - # print("卡号！！！！！！！！！！！") - iccid = sim.getIccid() - iccid = num_str_to_chinese(iccid)#转为中文 - if iccid != -1 and len(iccid) > 5: - print("iccid{}".format(iccid)) - tts_play_str = "当前卡号为:{}".format(iccid) - else: - tts_play_str = "卡号查询失败" - EventMesh.publish("play_tts",[tts_play_str,0]) - elif "设备号" in msg: - imei = modem.getDevImei() - if imei != -1 and len(imei) > 5: - print("IMEI：{}".format(imei)) - tts_play_str = "当前设备号为:{}".format(imei) - else: - tts_play_str = "设备号查询失败" - EventMesh.publish("play_tts",[tts_play_str,0]) - elif "音量" in msg : - tts_play_str = "当前音量是{}".format(self.VC_TTS_volume) - EventMesh.publish("play_tts",[tts_play_str,0]) - elif "语速" in msg : - tts_play_str = "当前语速是{}".format(EventMesh.publish("find_rate")) - EventMesh.publish("play_tts",[tts_play_str,0]) - - #改变音色判断 - def change_voice(self,event ,msg): - voice = msg - # if "男声" in msg or "普通话" in msg: - # self.VC_TTS_voice_key = "男" - # else: - # self.VC_TTS_voice_key = voice - self.outbound_attributess_provider = "xun-fei" - if "男声" in msg or "普通话" in msg: - self.VC_TTS_voice_key = "男" - self.outbound_attributess_provider = "volcengine" - elif "女声" in msg: - self.VC_TTS_voice_key = "女" - self.outbound_attributess_provider = "volcengine" - elif "童声" in msg: - self.VC_TTS_voice_key = "ç«¥" - self.outbound_attributess_provider = "volcengine" - elif "香港粤语" in msg: - self.VC_TTS_voice_key = "香港粤语" - elif "四川话" in msg: - self.VC_TTS_voice_key = "四川" - elif "上海话" in msg: - self.VC_TTS_voice_key = "上海" - elif "安徽合肥话" in msg: - self.VC_TTS_voice_key = "安徽合肥" - elif "山东话" in msg: - self.VC_TTS_voice_key = "山东话" - elif "陕西话" in msg: - self.VC_TTS_voice_key = "陕西" - elif "广东话" in msg: - self.VC_TTS_voice_key = "广东" - elif "成都话" in msg: - self.VC_TTS_voice_key = "成都" - elif "湖南话" in msg: - self.VC_TTS_voice_key = "湖南" - elif "苏州话" in msg: - self.VC_TTS_voice_key = "苏州" - elif "东北话" in msg: - self.VC_TTS_voice_key = "东北" - elif "河南话" in msg: - self.VC_TTS_voice_key = "河南" - elif "湖北话" in msg: - self.VC_TTS_voice_key = "湖北" - elif "闽南话" in msg: - self.VC_TTS_voice_key = "闽南" - elif "内蒙古话" in msg: - self.VC_TTS_voice_key = "内蒙古话" - EventMesh.publish("set_storage_config",["VC_TTS_voice_key",self.VC_TTS_voice_key]) - tts_play_str = "声音已改为{}".format(voice) - EventMesh.publish("play_tts",[tts_play_str,0]) - - #闹钟的url播放接口 - def play_single_voice(self,event = None,msg= None): - EventMesh.publish("publish_AI_btnInfo",{"func":self.play_single_voice_cb,"data":msg}) - def play_single_voice_cb(self,data): - try: - new_voice_url = data[0] #音频地址 - is_close_VC = data[1] - if new_voice_url != "": - self.AI_new_voice_url = new_voice_url - self.alarm_play_flag = 1 - if is_close_VC != None: - self.Close_flag = is_close_VC - if self.AI_VC_send_data_queue.empty()!=True: - self.AI_VC_send_data_queue.clear() - if self.AI_ASR_send_data_queue.empty()!=True: - self.AI_ASR_send_data_queue.clear() - EventMesh.publish("set_led_state", {"ai_state": "thinking"}) - self.AI_ASR_send_buf_len = 0 - self._uplink_request_stop("play_single_voice_cb") - self.websocket_deconnect_VC() - self.AI_VC_start() - self.play_new_voice_flag = 1 - # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - except Exception as e: - usys.print_exception(e) - pass - #语音留言 - def play_new_voice(self,event = None,msg= None): - EventMesh.publish("publish_AI_btnInfo",{"func":self.play_new_voice_cb,"data":msg}) - def play_new_voice_cb(self,data): - try: - new_voice_url = data[0] #音频地址 - is_close_VC = data[1] - if new_voice_url != "": - self.recv_voice = 1#收到留言 - print("已收到最新留言！！！！！") - # EventMesh.publish("set_motor_time",[1,1,5,8,4])#设置马达参数 - self.AI_new_voice_url = new_voice_url - # if self.pwk_cnt_voice == 0 and self.pwk_notice_flag == 0:#并且不在多条留言的状态 - # EventMesh.publish("play_tts",["您有一条语音留言，10秒内按下中间键可以收听",1]) - if self.pwk_notice_flag == 0 and EventMesh.publish("get_app_new_voice_flag"):#不处于播放状态并且是小程序下发的语音而不是请求的 - self.pwk_cnt_voice += 1#加上当前这条 - EventMesh.publish("set_storage_config",["is_noread_voice",1]) - EventMesh.publish("set_app_new_voice_flag",0) - print("voice_add_to = {}".format(self.pwk_cnt_voice)) - print("self.pwk_notice_flag = {}".format(self.pwk_notice_flag)) - if self.pwk_cnt_voice >1: - if self.pwk_notice_flag == 0: - tts_str = "您有" + str(self.pwk_cnt_voice) + "条未读语音留言，10秒内按下中间键可以收听" - EventMesh.publish("play_tts",[tts_str,1]) - elif self.pwk_cnt_voice == 1: - if self.pwk_notice_flag == 0: - tts_str = "您有1条语音留言，10秒内按下中间键可以收听" - EventMesh.publish("play_tts",[tts_str,1]) - if is_close_VC != None: - self.Close_flag = is_close_VC - if self.AI_VC_send_data_queue.empty()!=True: - self.AI_VC_send_data_queue.clear() - if self.AI_ASR_send_data_queue.empty()!=True: - self.AI_ASR_send_data_queue.clear() - EventMesh.publish("set_led_state", {"ai_state": "thinking"}) - self.AI_ASR_send_buf_len = 0 - self._uplink_request_stop("play_new_voice_cb") - self.websocket_deconnect_VC() - self.AI_VC_start() - self.play_new_voice_flag = 1 - # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - except Exception as e: - usys.print_exception(e) - pass - #历史留言查看 - def play_his_voice(self,event = None,msg= None): - EventMesh.publish("publish_AI_btnInfo",{"func":self.play_his_voice_cb,"data":msg}) - def play_his_voice_cb(self,data): - try: - new_voice_url = data[0] #音频地址 - is_close_VC = data[1] - time = data[2] - tts_play_str = time + "收到" - EventMesh.publish("play_tts",[tts_play_str,1]) - if new_voice_url != "": - self.recv_voice = 1#收到留言 - print("历史留言查看") - # tts_play_str = "收到" - # EventMesh.publish("play_tts",[tts_play_str,1]) - # EventMesh.publish("set_motor_time",[1,1,5,8,4])#设置马达参数 - self.AI_new_voice_url = new_voice_url - if is_close_VC != None: - self.Close_flag = is_close_VC - if self.AI_VC_send_data_queue.empty()!=True: - self.AI_VC_send_data_queue.clear() - if self.AI_ASR_send_data_queue.empty()!=True: - self.AI_ASR_send_data_queue.clear() - EventMesh.publish("set_led_state", {"ai_state": "thinking"}) - self.AI_ASR_send_buf_len = 0 - self._uplink_request_stop("play_his_voice_cb") - self.websocket_deconnect_VC() - self.AI_VC_start() - self.play_new_voice_flag = 1 - # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - except Exception as e: - usys.print_exception(e) - pass - #等待未读留言查看 - def play_wait_voice(self,event = None,msg= None): - EventMesh.publish("publish_AI_btnInfo",{"func":self.play_wait_voice_cb,"data":msg}) - def play_wait_voice_cb(self,data): - try: - new_voice_url = data[0] #音频地址 - is_close_VC = data[1] - print("是否为第一次：{}".format(self.wait_first_flag)) - if self.wait_first_flag == 0: - tts_play_str = "您有" + str(len(EventMesh.publish("get_play_new_voice_list"))) + "条未听留言，10秒内按下中间键可以收听" - EventMesh.publish("play_tts",[tts_play_str,1]) - # self.wait_first_timer.start(1000,1,self.noread_wait_vocie) - self.wait_first_flag = 1 - elif self.wait_first_flag == 1: - self.wait_sort += 1 - self.AI_TTS_str = "第"+str(self.wait_sort)+"条" - if new_voice_url != "": - self.recv_voice = 1#收到留言 - print("未读留言查看") - self.AI_new_voice_url = new_voice_url - if is_close_VC != None: - self.Close_flag = is_close_VC - if self.AI_VC_send_data_queue.empty()!=True: - self.AI_VC_send_data_queue.clear() - if self.AI_ASR_send_data_queue.empty()!=True: - self.AI_ASR_send_data_queue.clear() - EventMesh.publish("set_led_state", {"ai_state": "thinking"}) - self.AI_ASR_send_buf_len = 0 - self._uplink_request_stop("play_wait_voice_cb") - self.websocket_deconnect_VC() - self.AI_VC_start() - self.play_new_voice_flag = 1 - # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - except Exception as e: - usys.print_exception(e) - pass - def noread_wait_vocie(self,*args):#如果一直没读 - self.wait_first_cnt += 1 - print("未读留言等待收听中：{}".format(self.wait_first_cnt)) - if EventMesh.publish("get_recv_voice") == 1:#如果检测到留言播放 - self.wait_first_cnt = 0 - self.wait_first_timer.stop() - if self.wait_first_cnt == 15: - EventMesh.publish("set_wait_first_flag",0) - EventMesh.publish("set_history_voice_flag",0) - EventMesh.publish("set_wait_voice_flag",0) - self.wait_first_cnt = 0 - self.wait_first_timer.stop() - #音乐播放 - def play_music(self,event = None,msg= None): - EventMesh.publish("publish_AI_btnInfo",{"func":self.play_music_cb,"data":msg}) - def play_music_cb(self,data): - try: - music_name = data[0] - new_voice_url = data[1] #音频地址 - is_close_VC = data[2] - if new_voice_url != "": - self.AI_new_voice_url = new_voice_url - self.music_play_flag = 1#表示音乐播放开启 - EventMesh.publish("play_tts",["下面为您播放歌曲{}".format(music_name),1]) - if is_close_VC != None: - self.Close_flag = is_close_VC - if self.AI_VC_send_data_queue.empty()!=True: - self.AI_VC_send_data_queue.clear() - if self.AI_ASR_send_data_queue.empty()!=True: - self.AI_ASR_send_data_queue.clear() - EventMesh.publish("set_led_state", {"ai_state": "thinking"}) - self.AI_ASR_send_buf_len = 0 - self._uplink_request_stop("play_music_cb") - self.websocket_deconnect_VC() - self.AI_VC_start() - self.play_new_voice_flag = 1 - # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - except Exception as e: - usys.print_exception(e) - - #AI翻译接收TTS - def play_trans_tts(self,event = None,msg= None): - EventMesh.publish("publish_AI_btnInfo",{"func":self.play_trans_tts_cb,"data":msg}) - def play_trans_tts_cb(self,data): - try: - new_voice_url = data #音频地址 - if new_voice_url != "": - self.AI_new_voice_url = new_voice_url - if self.AI_VC_send_data_queue.empty()!=True: - self.AI_VC_send_data_queue.clear() - if self.AI_ASR_send_data_queue.empty()!=True: - self.AI_ASR_send_data_queue.clear() - EventMesh.publish("set_led_state", {"ai_state": "thinking"}) - self.AI_ASR_send_buf_len = 0 - self._uplink_request_stop("play_trans_tts_cb") - self.websocket_deconnect_VC() - self.AI_VC_start() - EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 - self.play_new_voice_flag = 1 - # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) - # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) - except Exception as e: - usys.print_exception(e) - def AI_client_new_voice(self): #上传未读的语音地址/包括音频地址也可以 - json_data = \ - { - "event": "audio.convert", - "payload": { - "audio_convert_request": { - "audio_url": self.AI_new_voice_url, - "bit_depth": 16, - "channels": 1, - "sample_rate": 16000, - "target_format": "pcm" - } - } - } - self.AI_new_voice_url = "" - print("AI_client_new_voice!!!!!!!!!!!!!!!!!!!!!!!!!!!!!{}".format(json_data)) - self.AI_client_VC_send(ujson.dumps(json_data)) - - - def chenggong_tone(self): - EventMesh.publish("audio_montage_play",["chenggong.mp3"]) - # self.aud.aud_tone_play(10,400) - def send_tone(self): - EventMesh.publish("audio_montage_play",["send_tone.mp3"]) - def concurrency_limit_exceeded_tone(self): - self.Close() - EventMesh.publish("audio_montage_play",["chaobingfatis.mp3"]) #播放超并发提醒 - #给服务器回应ack - - - ############################################ - #关闭音频播放线程 - def AI_audio_queue_thread_close(self): - print("AI_audio_queue_thread_close1!!!!!!!!!!!!!!!!!!!!!!") - if self.AI_audio_queue_thread_id != 0: - count =500 - while True: - count -= 1 - if self.AI_audio_queue_thread_lock == 0 or count <=0: - break - else: - self.AI_audio_queue_thread_lock = 2 - print("AI_audio_queue_thread going:{}...".format(count)) - utime.sleep_ms(1) - self.AI_audio_queue_thread_lock = 0 - print("AI_audio_queue_thread_close2!!!!!!!!!!!!!!!!!") - if self.AI_audio_queue_thread_id != 0: - try: - _thread.stop_thread(self.AI_audio_queue_thread_id) - except Exception as e: - usys.print_exception(e) - self.AI_audio_queue_thread_id = 0 - # 音频队列 - def AI_audio_queue_thread(self): - print("AI_audio_queue_thread start") - count = 0 - if self.AI_audio_queue.empty()!=True: - self.AI_audio_queue.clear() - print("音频数据还有剩余！！！！！！！！！！！！") - while True: - try: - if self.AI_audio_queue_thread_lock == 2 or self.VC_stop_play_audio_flag == 0: - print("AI_audio_queue_thread 该退出了") - break - # - count+=1 - if count%500 == 1 and VC_HEARTBEAT_LOG: - print("AI_audio_queue_thread on") - utime.sleep_ms(1) - #print("AI_audio_queue_thread") - if self.AI_audio_queue.empty() : - self.AI_audio_queue_thread_lock = 0 - - - else: - self.AI_audio_queue_thread_lock = 1 - data = self.AI_audio_queue.get() - self.close_timerout = 0 - if self.play_flag == 0: - self.play_flag =1 - #self.AI_client_tts_data_len=len(data) - #print("audio_queue:len:{}".format(len(data[0]))) - - if len(data[0]) > 10: - try: - if self._ptt_should_drop_inbound_tts(): - continue - except Exception: - pass - for i in range(5): - if self.VC_Play_flag == 0: - self.VC_Play_Start() - utime.sleep_ms(2) - else: - break - # 打断后 PCM 偶发未打开：加重试，避免「队列有数据、play_pcm=None、无声」 - if self.VC_play_TYPE == VC_play_TYPE_PCM and self.play_pcm is None and self.VC_stop_play_audio_flag == 1: - for j in range(4): - if self.VC_Play_flag == 0: - self.VC_Play_Start() - utime.sleep_ms(30 + j * 35) - if self.play_pcm is not None: - break - try: - if self.VC_play_TYPE == VC_play_TYPE_PCM: - # if self.VC_play_samplerate == 8000: - # tmp= self.aud.playStream(1,data[0] ) - # else: - write_buf = bytearray(data[0]) - #print("play_pcm.stream_write1 len:{}".format(len(write_buf))) - if self.play_pcm != None: - #print("play_pcm.stream_write !!!") - ret = self.play_pcm.stream_write(write_buf,len(write_buf)) - #ret= self.aud.playStream(1, data[0])#播放音频流 - if self.tts_out_data_len == 0: - try: - self._tts_first_write_ms = utime.ticks_ms() - except Exception: - self._tts_first_write_ms = 0 - try: - sid = getattr(self, "_cur_sid", "") - did = getattr(self, "_cur_dialog_id", "") - except Exception: - sid, did = "", "" - print("[TTS] first_write ret={} len={} sid={} did={}".format( - ret, len(write_buf), sid, did - )) - if ret <= 0: - print("[TTS] stream_write ret={} len={} (no audio?)".format(ret, len(write_buf))) - self.tts_out_data_len += ret - try: - if getattr(self.fsm, "await_played", False) and self.AI_audio_queue.empty(): - self._vc_try_play_end_tick() - except Exception: - pass - #print("play_pcm.stream_write2:{}".format(self.tts_out_data_len)) - elif self.VC_play_TYPE == VC_play_TYPE_MP3: - tmp= self.aud.playStream(3, data[0])#播放音频流 - - if tmp != 0: - print("2playStream:error tmp:{} data len:{}!!!!!!!!!!!!".format(tmp,len(data[0]))) - # # #播放失败的时候填充WAV头 - # data_len = len(data[0]) - # data_len_b = data_len.to_bytes(4,'little')#数据长度转16进制数组 - # h_data = [ 0x52 ,0x49 ,0x46 ,0x46 #,0x24 ,0x00 ,0x00 ,0x00 #文件标识、文件大小 - # ,0x57 ,0x41 ,0x56 ,0x45 ,0x66 ,0x6D ,0x74 ,0x20 #文件格式、格式块标识符 - # ,0x10 ,0x00 ,0x00 ,0x00 ,0x01 ,0x00 ,0x01 ,0x00 ,0x80 ,0x3E ,0x00 ,0x00 #格式块大小4、音频格式2、声道数2、采样率4 - # ,0x80 ,0x3E ,0x00 ,0x00 ,0x02 ,0x00 ,0x10 ,0x00 ,0x64 ,0x61 ,0x74 ,0x61]#每秒字节数4、数据块对齐2、位深度2、data4 - # h_data_b = bytes(h_data) - # file_len = data_len+len(h_data_b)+4+4 - # file_len_b = file_len.to_bytes(4,'little')#文件长度转16进制数组 - # h_data_b = h_data_b[:4]+file_len_b+h_data_b[4:]+data_len_b - # #self.aud.playStream(2, h_data_b)#播放音频流 - # tmp= self.aud.playStream(2, h_data_b+data[0])#播放音频流 - # print("3playStream:tmp:{} !!!!!!!!!!!!".format(tmp)) - - - mem = gc.mem_free() - #print('1剩余内存RAM:{} KB'.format(mem / 1024)) - except Exception as e: - print("AI_audio_queue_thread:ERROR:") - usys.print_exception(e) - - # if self.VC_state == VC_State_TTSEnd and self.AI_audio_queue.empty() :#self.AI_audio_queue为空且tts下发结束 - # #通知服务器播放结束 - # self.AI_client_VC_LocalRespondingEnded() - except Exception as e: - print("AI_audio_queue_thread:ERROR:") - usys.print_exception(e) - self.AI_audio_queue_thread_lock = 0 - self.AI_audio_queue_thread_id = 0 - print("AI_audio_queue_thread:exit") - def factory_mode(self): - print("factory_mode") - EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) - EventMesh.publish("publish_AI_btnInfo",{"func":self.factory_start}) - def factory_start(self): - self.factory_timer.start(3000, 1, self.factory_timer_cb) - def factory_timer_cb(self, *args, **kwargs): - print("factory_timer_cb play......") - #self.aud.play(1,0,'U:/static/peizhichenggong.mp3')#切换成功 - EventMesh.publish("audio_montage_play",["xinhaoruo1.mp3"]) - def Close_VC(self,event,msg): - EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) - def Close(self): - try: - if getattr(self, "VC_mode", 0) != 1: - print("[VC][long_idle] Close() enter VC_mode={} {} VC_satrt_flag={} VC_record_state_flag={}".format( - getattr(self, "VC_mode", -1), self._vc_fsm_log_snip(), - getattr(self, "VC_satrt_flag", -1), getattr(self, "VC_record_state_flag", -1))) - except Exception: - pass - self.VC_stop_play_audio_flag = 0 - try: - self.AI_key_long_press_flag = 0 - except Exception: - pass - self.VC_Thinking_timerout = 0 - self.AI_ASR_timer.stop() - self._1S_timer.stop() - self.close_timerout = 0 - self._idle_deep_sec = 0 - self.factory_timer.stop() - self.AI_TTS_str = "" - self.AI_ASR_str = "" - self.AI_ASR_str_tmp = "" - self.AI_ASR_flag = 0 - self.AI_ASR_check_timeout = 0 - self.AI_ASR_error_timeout = 0 - self.LocalRespondingEnded_timerout = 0 - self._need_audio_played_ack = 0 - self._tts_pending = 0 - self._tts_pending_ms = 0 - self._vc_session_started = 0 - self._ptt_audio_created = 0 - self._vc_pending_interrupt_uplink = 0 - try: - self._skip_started_uplink_once = 0 - except Exception: - pass - try: - self._ptt_abort_interrupt_arm() - except Exception: - pass - # Power 短按停止对话后，再次按 AI 需要新建 dialog_id：清理本轮 dialog/session 标识 - try: - self._cur_dialog_id = "" - except Exception: - pass - try: - self._cur_sid = "" - except Exception: - pass - self.is_ruppt_tts = 0#打断标志位清零 - print("ai_VC Close1!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") - try: - if getattr(self, "VC_mode", 0) != 1: - print("[VC][long_idle] step=set_led_state_idle (first)") - except Exception: - pass - EventMesh.publish("set_led_state", {"ai_state": "idle"}) - self.tts_in_data_len = 0 - self.tts_out_data_len = 0 - #必须先销毁线程，不然线程会容易报错 - - try: - self.AI_audio_queue_thread_close()#销毁音频线程 - except Exception as e: - usys.print_exception(e) - try: - self.AI_VC_send_data_queue_thread_close()#销毁数据发送线程 - except Exception as e: - usys.print_exception(e) - - # Close 可能被不同线程触发：统一切回主线程执行 stop - try: - if getattr(self, "VC_mode", 0) != 1: - print("[VC][long_idle] step=schedule_uplink_stop reason=Close (async publish_AI_btnInfo)") - except Exception: - pass - EventMesh.publish("publish_AI_btnInfo",{"func":self._uplink_request_stop, "data":"Close"}) - - print("ai_VC Close1.1!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") - - print("ai_VC Close1.2!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") - - try: - if getattr(self, "VC_mode", 0) != 1: - print("[VC][long_idle] step=websocket_deconnect_VC") - except Exception: - pass - try: - self.websocket_deconnect_VC() - except Exception as e: - usys.print_exception(e) - - - - - - print("ai_VC Close1.3!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") - - print("ai_VC Close2!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") - - - - - if self.record_data_save_flag == 1: - if self.record_fd : - self.record_fd.close() - self.record_fd = None - if self.play_data_save_flag == 1: - if self.play_fd !=None: - self.play_fd.close() - self.play_fd = None - print("ai_VC Close3!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\r\n") - - if self.AI_VC_send_data_queue.empty()!=True: - self.AI_VC_send_data_queue.clear() - self.play_new_voice_flag = 0 #语音消息播放状态 - if self.voice_choose_flag == 0: # 如果没有未读语音 - self.AI_new_voice_url = "" # 语音播放地址 - print("关闭并重置语音标志位") - else: - print("关闭但是没有重置语音标志位") - self.AI_ASR_send_buf_len = 0 - #self.AI_WS_connect_Queue.put(self.AI_client_VC) - try: - if getattr(self, "VC_mode", 0) != 1: - print("[VC][long_idle] step=set_led_state_idle (second) -> VC_init VC_satrt_flag=0") - except Exception: - pass - EventMesh.publish("set_led_state", {"ai_state": "idle"}) - self.VC_init() - self.VC_satrt_flag = 0 - # Level2：会话结束后仍可继续 1s 节拍累计 idle 直至深睡（依赖 VC_IDLE_DEEP_SLEEP_SEC） - try: - _dsec = 60 - _den = 1 - if "vc_config" in globals() and vc_config: - _dsec = int(vc_config.get_int("VC_IDLE_DEEP_SLEEP_SEC", 60)) - _den = int(vc_config.get_int("VC_IDLE_DEEP_SLEEP_ENABLE", 1)) - if _den and _dsec > 0: - self._1S_timer.start(1000, 1, self._1S_timer_cb) - except Exception: - pass - # try: - # while True: - # if self.AI_WS_connect_Queue.empty()!=True: - # AI_WS_connect = self.AI_WS_connect_Queue.get() - # if AI_WS_connect : - # try: - # AI_WS_connect._close() - # AI_WS_connect = None - # except Exception as e: - # usys.print_exception(e) - # else: - # print("websocket链接已为空") - # else: - # break - # except Exception as e: - # usys.print_exception(e) - - def VC_init2(self): - self.VC_init() - - print("ai_VC VC_init2!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") - try: - self.websocket_deconnect_VC() - except Exception as e: - usys.print_exception(e) - print("ai_VC Close3!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\r\n") - self._1S_timer.stop() - if self.AI_VC_send_data_queue.empty()!=True: - self.AI_VC_send_data_queue.clear() - - self.AI_ASR_send_buf_len = 0 - if self.AI_audio_queue.empty()!=True: - self.AI_audio_queue.clear() - def VC_init(self): - _q0 = self.AI_audio_queue.size() - self.VC_Play_End() - if self.AI_audio_queue.empty()!=True: - self.AI_audio_queue.clear() - print("[VC] init audio_queue {} -> {}".format(_q0, self.AI_audio_queue.size())) - # num = EventMesh.publish("get_volum_size") - # if num == None: - # num = 8 - # self.aud.setVolume(num)#恢复原始音量 +# -*- coding: utf-8 -*- +"""语音对话编排：按键/PTT、AI_VC_start/Close、定时器与 FSM 编排。""" +from vc_session_imports import * # noqa: F401,F403 +import _thread +from vc_session_io import VCSessionIoMixin +from vc_session_media import VCSessionMediaMixin + +class AI_VC(VCSessionIoMixin, VCSessionMediaMixin, Abstract): + def _vc_fsm_log_snip(self): + try: + f = self.fsm + return "sess={} turn={} tts_rx={} await={} sp_up={}".format( + int(getattr(f, "session", -1)), + int(getattr(f, "turn", -1)), + int(getattr(f, "tts_rx_phase", -1)), + 1 if getattr(f, "await_played", False) else 0, + 1 if getattr(f, "speech_uplink_open", 0) else 0, + ) + except Exception: + return "fsm=?" + def _vc_capture_allowed(self, reason=""): + """采集/上行开麦前：下行播放或本地短音未结束则禁止。""" + try: + if self._vc_tts_pipeline_busy(): + return False + if self._vc_local_play_busy(): + return False + except Exception: + return False + return True + def _vc_stop_downlink_for_interrupt(self, reason="interrupt"): + """详设 DES-SCN-003：打断/取消时先本地停播清队列，再允许开麦。""" + try: + if getattr(self, "AI_audio_queue", None) is not None: + if self.AI_audio_queue.empty() != True: + self.AI_audio_queue.clear() + except Exception: + pass + try: + if self._vc_tts_pipeline_busy() or getattr(self, "is_ruppt_tts", 0) == 1: + self._drop_tts_audio = 1 + except Exception: + pass + try: + self._ptt_hard_mute_local_playback() + except Exception: + pass + try: + if getattr(self, "VC_Play_flag", 0) == 1 or getattr(self, "play_pcm", None) is not None: + self.VC_Play_End() + except Exception: + pass + try: + self._vc_local_play_active = 0 + self._vc_local_play_until_ms = 0 + except Exception: + pass + try: + self.fsm.await_played = False + self._need_audio_played_ack = 0 + self._tts_pending = 0 + self._tts_pending_ms = 0 + except Exception: + pass + try: + pending_intr = getattr(self, "_vc_pending_interrupt_uplink", 0) == 1 + if ( + not pending_intr + and getattr(self, "VC_record_state_flag", 0) != 0 + and getattr(self, "record_data_save_flag", 0) == 0 + ): + self._uplink_request_stop("stop_downlink/" + str(reason)) + except Exception: + pass + def _vc_long_idle_active_speech_or_later(self): + # 对应旧 VC_state >= SpeechIng + try: + t = getattr(self.fsm, "turn", VCTurnState.Idle) + return t not in (VCTurnState.Idle, VCTurnState.Listening) + except Exception: + return False + def _vc_is_business_idle(self): + # Level2：LED ai_state 为 idle，且无录音/流播/音乐闹钟等（与 WS ping 无关） + try: + if getattr(self, "music_play_flag", 0) == 1: + return False + if getattr(self, "alarm_play_flag", 0) == 1: + return False + except Exception: + pass + try: + if getattr(self, "VC_record_state_flag", 0) != 0: + return False + if getattr(self, "VC_Play_flag", 0) != 0: + return False + except Exception: + pass + try: + if EventMesh.publish("get_vc_stream_active") == 1: + return False + except Exception: + pass + try: + st = EventMesh.publish("get_led_debug_state") + if st and isinstance(st, dict): + if (st.get("ai") or "") != "idle": + return False + except Exception: + return False + return True + def _vc_idle_deep_sleep_impl(self): + try: + _need_close = (self.VC_satrt_flag == 1) or (getattr(self, "AI_client_VC_isconnect", 0) == 1) + print("[VC][idle_L2] 触发休眠流程 need_Close={} VC_mode={}".format( + 1 if _need_close else 0, getattr(self, "VC_mode", -1))) + except Exception: + pass + try: + if self.VC_satrt_flag == 1 or getattr(self, "AI_client_VC_isconnect", 0) == 1: + try: + print("[VC][idle_L2] 调用 Close 断开会话/链路") + except Exception: + pass + self.Close() + except Exception: + pass + try: + utime.sleep_ms(100) + except Exception: + pass + try: + print("[VC][idle_L2] 发布 Hsensor_set_sleep") + EventMesh.publish("Hsensor_set_sleep") + except Exception: + pass + try: + print("[VC][idle_L2] 发布 SYSTEM_SLEEP ({})".format(constants.EVENT.SYSTEM_SLEEP)) + EventMesh.publish(constants.EVENT.SYSTEM_SLEEP) + except Exception: + pass + def _vc_rx_resets_long_idle_timer(self, recv_data): + """是否应因本帧下行清零 close_timerout（排除保活/心跳，含 V2.5 文本 ping 对端应答）。""" + try: + if recv_data is None: + return False + if isinstance(recv_data, (bytes, bytearray)): + return True + if not isinstance(recv_data, str): + return False + t = recv_data.strip() + if not t: + return False + tl = t.lower() + if tl in ("ping", "pong"): + return False + try: + jd = ujson.loads(t) + except Exception: + return True + ev = jd.get("event") + if ev is None or ev == "": + ev = jd.get("type") + if ev is None: + return True + if str(ev).strip().lower() in _VC_KEEPALIVE_RX_EVENTS: + return False + return True + except Exception: + return True + def _handle_pwk_short_release(self, event=None, msg=None, merged_ai_tail=False): + """电源键短按：读留言标志、退会议/翻译；否则 Close，或单键模式下走 AI_key_release。""" + print("pwk_key_release") + if EventMesh.publish("get_recv_voice") == 1: + EventMesh.publish("update_play_new_voice_info") + self.recv_voice = 0 + EventMesh.publish("set_history_voice_flag", 0) + self.AI_new_voice_url = "" + if self.trans_flag == 1 or self.meet_flag == 1: + EventMesh.publish("exit_meet_mode") + EventMesh.publish("exit_trans_mode") + return + if merged_ai_tail: + self.AI_key_release(event, msg) + else: + EventMesh.publish("publish_AI_btnInfo", {"func": self.Close}) + EventMesh.publish("stop_sos_call") + def _ai_key_press_ptt_mode(self): + """单对话 PTT：短按仅停 TTS（不换 sid）；非播放态短按吞掉；长按由 AI_key_long_press→ptt_begin。""" + if self.VC_mode != 1: + return False + try: + playing = False + try: + t = getattr(self.fsm, "turn", VCTurnState.Idle) + playing = t in (VCTurnState.TtsReceiving, VCTurnState.Playing) + except Exception: + playing = False + if playing or self._vc_tts_pipeline_busy(): + self._ptt_short_press_stop = 1 + try: + self._ptt_hard_mute_local_playback() + except Exception: + pass + try: + self.VC_Play_End() + except Exception: + pass + try: + self.fsm.await_played = False + except Exception: + pass + try: + self._ptt_led_lock_listening = 0 + self._ptt_interrupt_active = 0 + self.is_ruppt_tts = 0 + except Exception: + pass + try: + self._vc_publish_led("idle", "ptt_short_press/stop_tts") + except Exception: + pass + return True + except Exception: + pass + return True + def _ai_key_press_long_dialogue_interrupt(self): + """长对话：TTS/播放忙或 turn==Thinking 时短按 → RequestToSpeak（先 cancel，canceled 后再开麦）。""" + if self.VC_mode != 0: + return False + try: + if getattr(self, "AI_client_VC_isconnect", 0) != 1 or getattr(self, "_vc_session_started", 0) != 1: + return False + except Exception: + return False + need_cancel = False + if self._vc_tts_pipeline_busy(): + need_cancel = True + else: + try: + if getattr(self.fsm, "turn", VCTurnState.Idle) == VCTurnState.Thinking: + need_cancel = True + except Exception: + pass + if not need_cancel: + return False + print("[KEY] long interrupt -> RequestToSpeak (cancel then uplink)") + self.is_ruppt_tts = 1 + try: + self._vc_pending_interrupt_uplink = 1 + except Exception: + pass + EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_RequestToSpeak}) + return True + def _ai_key_press_long_dialogue_wake(self): + """长对话未连接或未 started：直接 AI_VC_start，勿 Close+sleep。""" + if self.VC_mode != 0: + return False + if getattr(self, "voice_flag", 0) == 1 or getattr(self, "meet_flag", 0) == 1 or getattr(self, "trans_flag", 0) == 1: + return False + try: + if getattr(self, "AI_client_VC_isconnect", 0) == 1 and getattr(self, "_vc_session_started", 0) == 1: + return False + except Exception: + pass + print("[KEY] long wake AI_VC_start (no Close)") + try: + self.Close_flag = 0 + except Exception: + pass + self.get_config_timer_count = 0 + self.key_state = 1 + self.AI_TTS_str = "" + self.AI_ASR_str = "" + self.AI_ASR_str_tmp = "" + EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_VC_start}) + self.AI_key_long_press_flag = 0 + return True + def _vc_led_alert_rb_flash(self): + try: + EventMesh.publish("set_led_state", {"alert": "rb_flash_2"}) + except Exception: + pass + def _ai_key_press_long_dialogue_when_session_ready(self): + """长对话已连接且 session å·² started：非播放态直接开麦续聊（勿 idle 时发 cancel）。""" + if self.VC_mode != 0: + return False + try: + if getattr(self, "AI_client_VC_isconnect", 0) == 1 and getattr(self, "_vc_session_started", 0) == 1: + # 模式切换提示音会置 Close_flag=1；用户再短按须恢复可开麦 + try: + self.Close_flag = 0 + except Exception: + pass + self._vc_clear_stale_record_state("key.press/continue") + self.request_uplink_start("key.press/continue") + return True + except Exception: + pass + return False + def _ai_key_release_handle_ptt(self): + """单对话 PTT 松手：仅在上行录音中时 end_speech；禁止误触发 AI_VC_start（否则新 sid / cancel 竞态）。""" + if self.AI_key_long_press_flag == 1: + if getattr(self, "VC_record_state_flag", 0) != 0: + try: + gap = 200 + if "vc_config" in globals() and vc_config: + gap = int(vc_config.get_int("PTT_RELEASE_END_SPEECH_DELAY_MS", 200)) + if gap > 0: + utime.sleep_ms(gap) + except Exception: + pass + print("[PTT] release end_speech asr_str={!r} rec={}".format( + self.AI_ASR_str, getattr(self, "VC_record_state_flag", -1))) + self.AI_client_VC_end_speech() + else: + self.AI_key_long_press_flag = 0 + try: + if getattr(self, "_ptt_short_press_stop", 0) == 1: + self._ptt_short_press_stop = 0 + except Exception: + pass + print("[PTT] release_no_uplink_skip") + return + try: + if getattr(self, "_ptt_short_press_stop", 0) == 1: + self._ptt_short_press_stop = 0 + except Exception: + pass + print("[PTT] release_short_no_long") + def _ai_key_long_press_route(self, event=None, msg=None): + """长按：翻译先 Close 再开翻译录音；单对话→ptt_begin；长对话→与短按一致走 AI_key_press 链路。""" + if self.trans_flag == 1: + self.Close() + EventMesh.publish("start_voice_record", 4) + self.AI_key_long_press_flag = 1 + if self.VC_mode == 1: + EventMesh.publish("publish_AI_btnInfo", {"func": self.ptt_begin}) + else: + EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_key_press}) + def _ptt_end_led_suppress(self): + # 保持 _vc_play_end_suppress_idle_led / _ptt_led_lock_listening 直至真开录（VC_Speech_Start） + pass + def _ptt_should_drop_inbound_tts(self): + """单对话打断窗口：丢弃迟到 TTS 帧 / 禁止起播。""" + try: + if getattr(self, "VC_mode", -1) != 1: + return False + if getattr(self, "_drop_tts_audio", 0) == 1: + return True + if getattr(self, "is_ruppt_tts", 0) == 1: + return True + if getattr(self, "_ptt_interrupt_active", 0) == 1: + return True + if getattr(self, "_vc_serial_canceling", 0) == 1: + return True + if getattr(self, "_ptt_pending_arm", 0) == 1: + return True + except Exception: + pass + return False + def _ptt_should_skip_remote_playback(self): + return self._ptt_should_drop_inbound_tts() + def _ptt_enter_interrupt_arm(self): + try: + self._ptt_arm_epoch = int(getattr(self, "_ptt_arm_epoch", 0)) + 1 + self._ptt_interrupt_active = 1 + self._ptt_led_lock_listening = 1 + self._vc_play_end_suppress_idle_led = 1 + self._drop_tts_audio = 1 + self.is_ruppt_tts = 1 + except Exception: + pass + def _ptt_abort_interrupt_arm(self): + try: + self._ptt_interrupt_active = 0 + self._ptt_led_lock_listening = 0 + self._vc_play_end_suppress_idle_led = 0 + self._drop_tts_audio = 0 + self.is_ruppt_tts = 0 + except Exception: + pass + def _ptt_on_record_hardware_started(self): + """真开录后：允许新 TTS、放开灯锁；旧 TTS 已在打断期丢弃。""" + try: + self._ptt_interrupt_active = 0 + self._ptt_led_lock_listening = 0 + self._vc_play_end_suppress_idle_led = 0 + self._drop_tts_audio = 0 + self.is_ruppt_tts = 0 + except Exception: + pass + def _ptt_hard_mute_local_playback(self): + try: + self.VC_Play_flag = 0 + if getattr(self, "VC_play_TYPE", None) == VC_play_TYPE_PCM: + if getattr(self, "play_pcm", None) is not None: + try: + self.play_pcm.stream_stop() + self.play_pcm.close() + except Exception: + pass + self.play_pcm = None + try: + self.aud.stopPlayStream() + self.aud.stopAll() + except Exception: + pass + if getattr(self, "AI_audio_queue", None) is not None: + try: + if self.AI_audio_queue.empty() != True: + self.AI_audio_queue.clear() + except Exception: + pass + except Exception: + pass + try: + gap = 0 + if "vc_config" in globals() and vc_config: + gap = int(vc_config.get_int("PTT_POST_MUTE_MS", 30)) + if gap > 0: + utime.sleep_ms(gap) + except Exception: + pass + def battery_priority_mute_vc(self, event=None, msg=None): + """低电/关机提示音：短暂停 VC TTS，避免被流式播放挡住。""" + try: + self._ptt_hard_mute_local_playback() + except Exception: + pass + try: + if getattr(self, "VC_Play_flag", 0) == 1 or getattr(self, "play_pcm", None) is not None: + self.VC_Play_End() + except Exception: + pass + def _vc_enqueue_btn(self, func, data=None): + """统一经 publish_AI_btnInfo 入队，在 btnInfo_queue_thread 串行执行。""" + try: + if data is not None: + EventMesh.publish("publish_AI_btnInfo", {"func": func, "data": data}) + else: + EventMesh.publish("publish_AI_btnInfo", {"func": func}) + except Exception: + pass + def _vc_ptt_cancel_wait_ms(self): + try: + if "vc_config" in globals() and vc_config: + return int(vc_config.get_int("PTT_CANCEL_WAIT_MS", 2500)) + except Exception: + pass + return 2500 + def _vc_ptt_need_interrupt_wait(self): + """单对话：服务端仍在思考/收 TTS/播报时，开录前须先 cancel 并等 canceled。""" + try: + if getattr(self, "VC_mode", -1) != 1: + return False + turn = getattr(self.fsm, "turn", VCTurnState.Idle) + if self._vc_tts_pipeline_busy(): + return True + if turn in ( + VCTurnState.Thinking, VCTurnState.TtsReceiving, + VCTurnState.Playing, VCTurnState.AwaitPlayedAck, + ): + return True + except Exception: + pass + return False + def _vc_begin_ptt_cancel_wait(self, reason="ptt_interrupt"): + try: + self._ptt_pending_arm = 1 + self._vc_serial_canceling = 1 + try: + self._vc_cancel_deadline_ms = utime.ticks_add( + utime.ticks_ms(), self._vc_ptt_cancel_wait_ms()) + except Exception: + self._vc_cancel_deadline_ms = 0 + json_data = {"event": "cancel", "payload": {}} + self.AI_client_VC_send(ujson.dumps(json_data)) + try: + self.fsm.reset_rx_phases() + except Exception: + pass + self._drop_tts_audio = 1 + print("[VC][serial] cancel_wait reason={} turn={}".format( + reason, getattr(getattr(self, "fsm", None), "turn", "?"))) + except Exception: + pass + def _vc_on_ptt_cancel_settled(self, via="canceled"): + """btn 线程：cancel 已收尾，执行挂起的 PTT 开录。""" + try: + if not getattr(self, "_vc_serial_canceling", 0) and not getattr(self, "_ptt_pending_arm", 0): + return + self._vc_serial_canceling = 0 + self._vc_cancel_deadline_ms = 0 + if getattr(self, "_ptt_pending_arm", 0): + self._ptt_pending_arm = 0 + print("[VC][serial] ptt_arm after {}".format(via)) + self._ptt_arm_uplink() + except Exception: + pass + def _vc_tick_serial_cancel_timeout(self): + if not getattr(self, "_vc_serial_canceling", 0): + return + try: + dl = getattr(self, "_vc_cancel_deadline_ms", 0) + if dl and utime.ticks_diff(utime.ticks_ms(), dl) >= 0: + print("[VC][serial] cancel_wait timeout -> arm") + self._vc_on_ptt_cancel_settled(via="timeout") + except Exception: + pass + def _ptt_arm_uplink(self, t0=0): + """btn 线程：cancel 已结束或无需 cancel 时，发 audio.create 并开麦。""" + try: + self.is_ruppt_tts = 0 + except Exception: + pass + try: + self._ptt_audio_created = 0 + except Exception: + pass + try: + self.AI_ASR_flag = 0 + except Exception: + pass + try: + self.AI_client_VC_send_speech() + except Exception: + pass + try: + self.VC_Speech_Start_now() + except Exception: + pass + try: + self._vc_publish_led("listening", "ptt_begin/recording") + except Exception: + pass + self._ptt_end_led_suppress() + try: + if t0: + print("[PTT] record started (ms={})".format(utime.ticks_ms() - t0)) + except Exception: + pass + def ptt_begin(self, reason="key.long_press"): + try: + if self.VC_mode != 1: + return + except Exception: + return + + # PTT 新一轮开始：清理模式切换/提示音退出标志 + try: + self.Close_flag = 0 + self._ptt_await_server = 0 + except Exception: + pass + + t0 = 0 + try: + t0 = utime.ticks_ms() + except Exception: + t0 = 0 + + self._ptt_enter_interrupt_arm() + try: + self._vc_publish_led("listening", "ptt_begin/pre") + except Exception: + pass + + # 本地先硬静音再 VC_init，避免 DMA/队列尾包漏播 + try: + self._ptt_hard_mute_local_playback() + except Exception: + pass + try: + self.VC_init() + except Exception: + pass + try: + if getattr(self, "VC_Play_flag", 0) == 1 or getattr(self, "play_pcm", None) is not None: + self.VC_Play_End() + except Exception: + pass + try: + self._ptt_hard_mute_local_playback() + except Exception: + pass + + # 若上一轮录音尚未结束，先强制停掉，避免 record_thread 复用出错 + try: + if getattr(self, "VC_record_state_flag", 0) != 0: + self.VC_Speech_End() + except Exception: + pass + + # 已在等 canceled：只刷新「待开录」，勿重复 audio.create + try: + if getattr(self, "_vc_serial_canceling", 0): + self._ptt_pending_arm = 1 + print("[PTT] cancel in progress, refresh pending arm") + self._ptt_end_led_suppress() + return + except Exception: + pass + + # 会话未连接：先建链并发 Start，等待 started 后再自动开麦 + try: + if getattr(self, "AI_client_VC_isconnect", 0) != 1: + self._ptt_force_record_on_started = 1 + self.AI_VC_start() + return + except Exception: + self._ptt_force_record_on_started = 1 + try: + self.AI_VC_start() + except Exception: + pass + return + + # 会话还没 initialized：先发 Start，请求服务端进入 started + if getattr(self, "_vc_session_started", 0) != 1: + self._ptt_force_record_on_started = 1 + try: + try: + if t0: + print("[PTT] wait server.started (ms={})".format(utime.ticks_ms() - t0)) + except Exception: + pass + self.AI_client_VC_start_requset() + except Exception: + pass + return + + # 服务端仍在思考/TTS：先 cancel，在 btn 队列收到 canceled 后再开录（方案甲串行） + if self._vc_ptt_need_interrupt_wait(): + self._vc_begin_ptt_cancel_wait("ptt_begin") + return + + self._ptt_arm_uplink(t0) + def _vc_led_state_debug_enabled(self): + try: + if "vc_config" in globals() and vc_config: + return vc_config.get_bool("LED_STATE_DEBUG", True) + except Exception: + pass + return True + def _vc_led_should_standby_idle(self): + """未录音且非 TTS 播中：待机/提示音/模式切换/单对话轮间应灭灯（DES-L3 idle），勿被 FSM.turn=Listening 误导。""" + try: + if getattr(self, "VC_record_state_flag", 0) != 0: + return False + if self._vc_tts_playback_active(): + return False + if getattr(self, "_ptt_await_server", 0) == 1: + return False + t = getattr(self.fsm, "turn", VCTurnState.Idle) + if t in ( + VCTurnState.Thinking, + VCTurnState.TtsReceiving, + VCTurnState.Playing, + VCTurnState.AwaitPlayedAck, + ): + return False + if getattr(self, "Close_flag", 0) == 1: + return True + if getattr(self, "VC_mode", -1) == 1: + return True + k = getattr(self, "_last_tts_kind", "") or "" + if k in ("prompt", "boot_prompt", "mode_switch_long", "mode_switch_ptt"): + return True + except Exception: + pass + return False + def _vc_publish_led(self, ai_state, reason=""): + try: + if getattr(self, "_ptt_led_lock_listening", 0) == 1: + if ai_state != "listening": + if self._vc_led_state_debug_enabled(): + print("[LED] publish blocked ai={} reason={} (ptt_arm_lock)".format( + ai_state, reason)) + return + except Exception: + pass + try: + if self._vc_led_state_debug_enabled(): + print("[LED] publish ai={} reason={} VC_mode={} rec={} turn={}".format( + ai_state, reason, getattr(self, "VC_mode", -1), + getattr(self, "VC_record_state_flag", -1), + getattr(getattr(self, "fsm", None), "turn", "?"))) + except Exception: + pass + try: + EventMesh.publish("set_led_state", {"ai_state": ai_state}) + except Exception: + pass + def _vc_led_sync_from_fsm(self): + """按 FSM/真实 TTS 播放恢复 AI 灯；本地 aud.play 短音不参与。""" + try: + if getattr(self, "_ptt_led_lock_listening", 0) == 1: + if getattr(self, "VC_record_state_flag", 0) == 0: + return + except Exception: + pass + try: + if self._vc_led_should_standby_idle(): + self._vc_publish_led("idle", "fsm_sync/standby_idle") + return + except Exception: + pass + try: + if getattr(self, "VC_mode", -1) == 1 and getattr(self, "_ptt_await_server", 0) == 1: + self._vc_publish_led("thinking", "fsm_sync/ptt_await") + return + except Exception: + pass + try: + if self._vc_tts_playback_active(): + self._vc_publish_led("playing", "fsm_sync/tts") + return + except Exception: + pass + try: + t = getattr(self.fsm, "turn", VCTurnState.Idle) + if t in (VCTurnState.Thinking, VCTurnState.TtsReceiving): + self._vc_publish_led("thinking", "fsm_sync/thinking") + elif t in (VCTurnState.Playing, VCTurnState.AwaitPlayedAck): + # 服务端已标记播放态但本地 PCM 尚未起播：仍显示思考，勿误绿灯 + self._vc_publish_led("thinking", "fsm_sync/await_tts") + elif t in (VCTurnState.Listening, VCTurnState.SpeechEnded, VCTurnState.SpeechStreaming): + self._vc_publish_led("listening", "fsm_sync/listening") + else: + self._vc_publish_led("idle", "fsm_sync/idle") + except Exception: + try: + self._vc_publish_led("idle", "fsm_sync/fallback") + except Exception: + pass + def _fsm_apply_actions(self, actions): + try: + for a in actions: + if not a: + continue + if a[0] == "uplink_stop": + self.request_uplink_stop(a[1] if len(a) > 1 else "") + elif a[0] == "uplink_start": + self.request_uplink_start(a[1] if len(a) > 1 else "") + elif a[0] == "play_start": + if getattr(self, "VC_mode", -1) == 1: + if self._ptt_should_skip_remote_playback(): + if self._vc_led_state_debug_enabled(): + print("[VC][serial] skip play_start (ptt_interrupt)") + continue + try: + self.fsm.on_local_play_start() + except Exception: + pass + EventMesh.publish("publish_AI_btnInfo", {"func": self.VC_Play_Start}) + elif a[0] == "send_audio_played": + self.request_send_audio_played() + except Exception: + pass + def _print_hw_diag(self): + # 仅诊断使用：打印当前分支能看到的麦克/音频相关配置和运行时状态。 + # JSWM2 老分支没有 device_profile/_PROFILE，这里宽容处理 + try: + profile = _PROFILE if isinstance(_PROFILE, dict) else {} + except Exception: + profile = {} + dev_type_profile = profile.get("dev_type", None) + flags = profile.get("flags", {}) + hardware = profile.get("hardware", {}) + product_cfg = profile.get("product_cfg", {}) + vc_cfg = {} + try: + vc_cfg = product_cfg.get("vc", {}) + except Exception: + vc_cfg = {} + + print("[hw-diag] ===== AI_VC hardware diag =====") + print("[hw-diag] profile.dev_type =", dev_type_profile) + print("[hw-diag] profile.flags =", flags) + print("[hw-diag] profile.hardware =", hardware) + print("[hw-diag] product_cfg.vc =", vc_cfg) + + try: + print("[hw-diag] VC_record_samplerate =", self.VC_record_samplerate) + print("[hw-diag] VC_play_samplerate =", self.VC_play_samplerate) + except Exception: + pass + try: + print("[hw-diag] vad_silence_runtime =", self.vad_silence) + except Exception: + pass + try: + print("[hw-diag] record_data_save_flag =", self.record_data_save_flag) + print("[hw-diag] record_max_bytes =", self.record_max_bytes) + except Exception: + pass + + try: + g = self.record.gain_get() + except Exception as e: + g = "gain_get_error: {}".format(e) + print("[hw-diag] record.gain_get =", g) + try: + print("[hw-diag] type(record) =", type(self.record)) + print("[hw-diag] type(aud) =", type(self.aud)) + except Exception: + pass + print("[hw-diag] ===== end of hw-diag =====") + def __init__(self): + # import voiceCall + # voiceCall.callStart("10086") + # return + self.http_get_config_flag = 0 + if self.http_get_config_flag == 1: + try: + self.http_get_config()#获取/更新配置参数 + except Exception as e: + usys.print_exception(e) + self.VC_30S_timer = osTimer() + self.ai_key_press_audio = osTimer() + self.VC_30S_timer.start(30000, 1, self.VC_30S_timer_cb) + self.online_kws = 0#是否开启线上关键词 + self.get_config_timer_count = 0 + self.record_data_save_flag = 0 #是否保存录音文件 + self.play_data_save_flag = 0#是否保存tts数据到文件 + self.record_fd = None + self.play_fd = None + self.is_VC_ACK = 0 #是否开启ack应答模式 + self.is_long_record = 0 #是否开启长录音模式（语音打断模式） + self.ai_OTA = OTA() + self.is_auth = 1 #是否开启dmp鉴权 + #self.VC_mode = 1 #0：短按连续问答模式 1：长按录音模式 + self.VC_mode = EventMesh.publish("get_storage_config","VC_mode")# + if self.VC_mode == None: + self.VC_mode = 0 + try: + self.VC_mode = int(self.VC_mode) + except Exception: + self.VC_mode = 0 + self.is_up_error= 0 #是否上传错误id + self.Close_flag =0 #=1时，连续对话播放完音频就退出 + # 服务端 VAD 结束标志：收到 vad.silence 后置 1，用于硬闸门禁止继续上行音频 + # （避免仍持续上行导致服务端推迟/抑制后续 TTS，从而一直停留在思考态蓝绿闪） + self._server_vad_silence = 0 + # 单轮状态标志：用于判定 completed 是否应自动进入下一轮 + self._turn_has_asr_completed = 0 + self._turn_has_vad_silence = 0 + self._turn_has_tts_started = 0 + # VC 会话是否已进入 started（服务端 session initialized） + # 重要：在收到 started 前发送音频帧会触发服务端 "session not initialized" → failed → 端侧重连死循环 + self._vc_session_started = 0 + # 开机/提示音：本轮 started 不自动开麦（避免问候 TTS 前误进 listening） + self._skip_started_uplink_once = 0 + self._last_tts_kind = "" + self._pending_tts_kind = "" + # Push-To-Talk（VAD关闭/单对话）是否已发送 audio.create + # 重要：协议要求 disable_vad=true 时必须 audio.create→音频帧→audio.completed + self._ptt_audio_created = 0 + # PTT：松手等服务器期间保持 thinking 勿回蓝闪聆听 + self._ptt_await_server = 0 + # 方案甲 btnInfo 串行：打断时先 cancel，收到 canceled（或超时）后再 audio.create + self._vc_serial_canceling = 0 + self._ptt_pending_arm = 0 + self._vc_cancel_deadline_ms = 0 + # PTT 打断：轮次号 + 开录前丢弃旧 TTS / 灯只显示聆听 + self._ptt_arm_epoch = 0 + self._ptt_interrupt_active = 0 + self._ptt_led_lock_listening = 0 + # 端侧主动发 synthesis 后到收到 tts.started 之间的窗口标志，用于阻止误开下一轮录音上行 + self._tts_pending = 0 + self._tts_pending_ms = 0 + # 各类兜底的节流时间戳（避免“兜底过激/连续触发”） + self._fb_last_thinking_restart_ms = 0 + self._fb_last_tts_pending_timeout_ms = 0 + # thinking 兜底：部分情况下服务端只返回 llm.completed 而不触发 tts.started/completed， + # 会导致一直停留在 thinking。此时超时自动回到聆听。 + self._llm_completed_ms = 0 + # 语音上行统计（真实发送到 VC 发送队列的开始/结束） + self._uplink_active = 0 + self._uplink_bytes = 0 + self._uplink_start_ms = 0 + + # 状态机骨架（后续逐步把旧逻辑迁移进来） + self.fsm = VCStateMachine(self) + + self.play_new_voice_flag =0 #播放文字留言flag + + self.wait_first_flag = 0#是否进入未读判定（只播报一次） + self.wait_first_timer = osTimer()#如果播放未读留言的时候未读则不要播放了 + self.wait_first_cnt = 0 + + self.wait_sort = 0#未读留言正在播第几条 + EventMesh.subscribe("play_tts",self.play_tts) + EventMesh.subscribe("play_new_voice",self.play_new_voice) + EventMesh.subscribe("play_his_voice",self.play_his_voice) + EventMesh.subscribe("play_wait_voice",self.play_wait_voice) + EventMesh.subscribe("play_single_voice",self.play_single_voice)#不带tts的播放（闹钟） + EventMesh.subscribe("play_music",self.play_music)#播放音乐 + EventMesh.subscribe("play_trans_tts",self.play_trans_tts)#接收翻译tts + EventMesh.subscribe("set_vad_silence",self.set_vad_silence) + EventMesh.subscribe("set_wait_first_flag",self.set_wait_first_flag) + EventMesh.subscribe("kws_even",self.kws_even) + EventMesh.subscribe("change_volume",self.change_volume) + EventMesh.subscribe("find_cmd",self.find_cmd) #查询命令 + EventMesh.subscribe("NLP_set_vad_silence",self.NLP_set_vad_silence) #设置聆听间隔 + EventMesh.subscribe("change_voice",self.change_voice) #改变音色 + EventMesh.subscribe("Close_VC",self.Close_VC) #外部结束对话 + EventMesh.subscribe("get_recv_voice",self.get_recv_voice) #获取留言播放状态 + EventMesh.subscribe("set_recv_voice",self.set_recv_voice) #设置留言播放状态 + self.voice_timer = osTimer()#语音留言10S按下按键确定是否播放的判断定时器 + self.voice_cnt = 0 #留言定时器计数 + self.voicerupt_flag = 0#留言选择按键接听标志位 + self.wait_voicerupt_flag = 0#未读留言选择按键接听标志位 + self.recv_voice = 0 #是否收到留言 + self.voice_notice = 0 #收到留言并播放则置为1，接下来要播放提示词 + self.voice_choose_flag = 0#当tts和语音留言冲突时，为了防止语音地址被误清，做标志位在close做判断 + + self.revoice_timer = osTimer()#语音留言10S按下按键确定是否回复的判断定时器 + self.revoice_cnt = 0 #回复留言定时器计数 + self.revoicerupt_flag = 0#留言选择按键接听标志位 + + self.music_timer = osTimer()#音乐播放定时器，等待tts播完才放音乐 + self.music_cnt = 0 #留言定时器计数 + + self.his_timer = osTimer()#历史留言播放定时器，等待tts播完再播留言 + self.his_cnt = 0#历史留言定时器技术 + + self.wait_timer = osTimer()#未读留言播放定时器，等待tts播完再播留言 + self.wait_cnt = 0#未读留言定时器技术 + + self.error_data = [] + self.asr_taskid = "" + self.VC_taskid = "" + self.VC_server = 2 #0柒牌正式服务器，1：测试服务器1,2：测试服务器2 + self.voice_flag = 0 + + #会议记录标志 + self.meet_flag = 0 + #AI翻译标志 + self.trans_flag = 0 + + #历史留言查看判断标志 + self.history_voice_flag = 0#公共变量判断 + #60S等待留言判断标志 + self.wait_voice_flag = 0#公共变量判断 + EventMesh.publish("AI_alarm_open")# + self.server_ip = EventMesh.publish("get_storage_config","server_ip")# + if self.server_ip == None: + if self.VC_server == 1:# + self.server_ip="118.145.158.207:22083" + elif self.VC_server == 2:# + self.server_ip="118.145.158.207:22082" + elif self.VC_server == 4:# + self.server_ip="118.145.158.207:22084" + elif self.VC_server == 5:# + self.server_ip="amp-wss.raisound.com" + elif self.VC_server == 6:# + self.server_ip="118.145.158.207:22080" + elif self.VC_server == 7:# + self.server_ip="118.145.158.207:22080" + elif self.VC_server == 8:# + self.server_ip="115.190.94.208:22080" + elif self.VC_server == 86:# + self.server_ip="118.145.158.207:22086" + else: + #self.server_ip="sevenpin.qipai.com.cn" + self.server_ip="118.145.158.207:22082" + # 默认走 constants.AMP_SERVER（由 default_srv.py 统一管理，可在 usr 覆盖） + try: + if constants.AMP_SERVER: + self.server_ip = constants.AMP_SERVER + except Exception: + pass + + #开机提示音 + self.power_on_tone = EventMesh.publish("get_storage_config","power_on_tone")# + try: + if "vc_config" in globals() and vc_config: + if not vc_config.get_bool("POWER_ON_TONE_ENABLED", True): + self.power_on_tone = None + except Exception: + pass + if self.power_on_tone == None: + try: + if "vc_config" in globals() and vc_config: + self.power_on_tone = vc_config.get_attr("POWER_ON_TONE_DEFAULT", None) + except Exception: + pass + + self.VC_record_TYPE = VC_record_TYPE_PCM #0:PCM 1:AMR + self.VC_play_TYPE = VC_play_TYPE_PCM #0:PCM 1:mp3 + self.VC_record_samplerate = 16000 + self.VC_play_samplerate = 16000 + self.VAD_data_len = 640 #10ms数据 + self.is_VAD = 0#=1打开vad打断 + # ARG_sample_rate, // 采样率 + # ARG_channel, // 通道数 + # ARG_frame_size, // 每帧字节数 + # ARG_energy_threshold, // 能量阈值 + # ARG_voice_time, // 语音持续时间阈值(ms) + # ARG_silence_time, // 静音持续时间阈值(ms) + # ARG_silence_start_threshold, // 静音开始阈值(ms) + self.vad = None + if self.is_VAD == 1: + self.vad = VAD.VAD(self.VC_record_samplerate ,1,self.VAD_data_len,600,100,300,100) + self.speech_rate = EventMesh.publish("get_storage_config","speech_rate")#语速配置 + if self.speech_rate == None: + self.speech_rate = 0 + + self.vad_silence = EventMesh.publish("get_storage_config","vad_silence")#语速配置 + if self.vad_silence == None: + try: + if "vc_config" in globals() and vc_config: + self.vad_silence = vc_config.get_int("VAD_SILENCE_DEFAULT", 300) + else: + self.vad_silence = 300 + except Exception: + self.vad_silence = 300 + + self.VC_TTS_voice_type = EventMesh.publish("get_storage_config","VC_TTS_voice_type")#音色配置 + if self.VC_TTS_voice_type == None : + try: + if "vc_config" in globals() and vc_config: + cfg = vc_config.get_attr("VC_TTS_VOICE_TYPE_DEFAULT", None) + if isinstance(cfg, dict): + self.VC_TTS_voice_type = cfg + except Exception: + pass + # 若配置未提供，沿用原逻辑内置兜底（保持兼容） + if self.VC_TTS_voice_type == None: + self.VC_TTS_voice_type = { + "女":{"bot_name":"柒妹妹","voice":"zh_female_wanwanxiaohe_moon_bigtts","lang":"zh-cn"}, + "男":{"bot_name":"柒先生","voice":"zh_male_wennuanahu_moon_bigtts","lang":"zh-cn"}, + "ç«¥":{"bot_name":"柒乐乐","voice":"ICL_zh_female_huoponvhai_tob","lang":"zh-cn"}, + } + + # self.VC_TTS_voice_type = {0:"zh_female_xinlingjitang_moon_bigtts",1:"zh_male_wennuanahu_moon_bigtts",2:"ICL_zh_female_huoponvhai_tob", + # 3:"aliyun::longjiaxin",4:"zh_female_daimengchuanmei_moon_bigtts"} + #self.VC_TTS_voice_type = ["aliyun::longxiaoxia","aliyun::longnan","aliyun::longling"] + + #EventMesh.publish("set_storage_config",["VC_TTS_voice_type",self.VC_TTS_voice_type]) + + try: + self.VC_TTS_voice_key = EventMesh.publish("get_storage_config","VC_TTS_voice_key")#音色配置 + if self.VC_TTS_voice_key == None : + try: + if "vc_config" in globals() and vc_config: + self.VC_TTS_voice_key = vc_config.get_attr("VC_TTS_VOICE_KEY_DEFAULT", None) + except Exception: + pass + if not self.VC_TTS_voice_key: + self.VC_TTS_voice_key = "男" + if self.VC_TTS_voice_key in self.VC_TTS_voice_type: + pass + else: + self.VC_TTS_voice_key = list(self.VC_TTS_voice_type.keys())[0] + print("音色配置：{}:{}\r\n\r\n".format(self.VC_TTS_voice_key,self.VC_TTS_voice_type[self.VC_TTS_voice_key])) + self.VC_TTS_volume = EventMesh.publish("get_storage_config","VC_TTS_volume")#音量配置 + except Exception as e: + usys.print_exception(e) + #配置词 + self.commands_mode = 2 #=1用大模型判断命令，=0用服务器本地插件判断命令 =2用设备本地判断方式 + self.commands = [] + if self.VC_TTS_volume == None: + self.VC_TTS_volume = 100 + #可ota配置的参数变量 + self.emotion = EventMesh.publish("get_storage_config","emotion")#情绪 + if self.emotion == None: + self.emotion = "" + self.ota_update_notice = EventMesh.publish("get_storage_config","ota_update_notice")#OTA升级提示语 + if self.ota_update_notice == None: + self.ota_update_notice = "" + self.AI_audio_queue_thread_id = 0 + self.AI_audio_queue_thread_lock = 0 + self.AI_VC_send_data_queue_thread_id = 0 + self.AI_VC_send_data_queue_thread_lock = 0 + self.AI_ASR_send_data_queue_thread_id = 0 + self.AI_ASR_send_data_queue_thread_lock = 0 + self.AI_record_tmp_queue = Queue(100) + self.AI_ASR_send_data_queue = Queue(200) + self.AI_VC_send_data_queue = Queue(100) + self.AI_audio_queue = Queue(100) + #self.AI_WS_connect_Queue = Queue(100) #释放用 + self.AI_TTS_str = "" + self.AI_ASR_str = "" + self.AI_ASR_flag = 0 + self.AI_ASR_str_tmp = "" + self.AI_ASR_send_buf_len = 0 + self.AI_ASR_text_tmp = "" + self.AI_ASR_text_repet_count = 0 + # #开始线程 + # if self.AI_audio_queue_thread_id ==0: + # self.AI_audio_queue_thread_id = _thread.start_new_thread(self.AI_audio_queue_thread, ()) + # else: + # print("AI_VC_start :AI_audio_queue_thread_id != 0 ") + # if self.AI_VC_send_data_queue_thread_id == 0: + # self.AI_VC_send_data_queue_thread_id = _thread.start_new_thread(self.AI_VC_send_data_queue_thread, ()) + + self.VC_dialog_id = ""# + self.VC_message_id=None + self.VC_task_id =None + self.AI_client_VC = None + self.AI_client_VC_recv_id = 0 + self.AI_client_VC_recv_lock = 0 + self.AI_client_VC_isconnect =0 + self.AI_client_VC_debug =0 + + self.asr_state = 0 # 预留：ASR 侧状态（当前未参与分支） + self.key_state = 0 + + self.AI_client_asr = None + self.AI_client_asr_recv_id = 0 + self.AI_client_asr_recv_lock = 0 + self.AI_client_asr_isconnect =0 + self.AI_ASR_timer = osTimer() + self.AI_ASR_timer_count = 0 + self.AI_client_asr_debug =0 + self.AI_ASR_check_timeout = 0 + self.AI_ASR_error_timeout = 0 + self.VC_Thinking_timerout = 0 + + + self.record_pcm = None + self.play_pcm = None + self.VC_record_thread_id = 0 + self.VC_record_thread_lock = 0 + self.VC_pingping_timerout = 0 + self.asr_pingping_timerout = 0 + self.factory_timer = osTimer() + self._1S_timer = osTimer() + self.Audio_buf_size = 20*1024 + self.record = audio.Record() + #self.record.end_callback(self.record_callback) + self.record.gain_set(4,10) + #self.record.gain_set(4,12) + self.aud = audio.Audio(0) + self.aud.setCallback(self.audio_cb) + # 确保 record/aud 已创建后再打印硬件/音频诊断信息（需要 gain_get/type(record)） + try: + self._print_hw_diag() + except Exception as e: + usys.print_exception(e) + #self.aud.setVolume(self.VC_TTS_volume //10) + self.audio_play_state = 1 #音频播放完状态 + self.close_timerout = 0 + self._idle_deep_sec = 0 + self._vc_idle_deep_armed_suppress = 0 + self.LocalRespondingEnded_timerout = 0 + + self.VC_Play_flag = 0 + self.play_flag = 0 + self.VC_record_state_flag = 0 + self.AI_key_long_press_flag = 0 + self._asr_completed_tone_done = 0 # asr.completed 本地 think 提示每轮仅一次 + self._last_asr_result_text = "" # 保留最近 asr.result，防 VC_Speech_End 早于 asr.completed 清空 + self._vc_last_auto_uplink_ms = 0 # 长对话自动开麦防抖 + self._vc_local_play_active = 0 # aud.play 本地短音占用（与采集互斥） + self._vc_local_play_until_ms = 0 + self._vc_pending_interrupt_uplink = 0 # 长对话短按打断：等 canceled 再开麦 + self.websocket_error_count = 0 + self.AI_new_voice_url = "" #未读语音地址/音乐播放地址/闹钟播放标志 + self.music_play_flag = 0 #音乐播放标志 + self.alarm_play_flag = 0 #闹钟播放标志 + self.tts_in_data_len = 0 + self.tts_out_data_len = 0 + self._tts_first_write_ms = 0 + self._tts_completed_wall_ms = 0 + self._tts_play_check_log_ms = 0 + self._need_audio_played_ack = 0#tts.completed 后需在本地播报结束后上报 audio.played；勿用 tts_in/out：tts.completed 会再调 VC_Play_Start 把二者清零 + self.VC_satrt_flag = 0 + self.VC_stop_play_audio_flag = 0 + self.VC_satrt_kws_flag = 0 + self.session_id = "" + self.stop_voice_timer=osTimer() + self.pwk_cnt_voice = 0 #不听语音后，积攒的次数，需要一起播放 + self.pwk_notice_flag = 0#是否处于积攒留言状态(用来实现单次提醒) + self.is_noread_voice = EventMesh.publish("get_storage_config","is_noread_voice") + if self.is_noread_voice == None: + self.is_noread_voice = 0#是否还有未读留言（开机判断） + self.is_ruppt_tts = 0#是否已经打断tts播放 + EventMesh.subscribe("AI_key_press", self.AI_key_press) + EventMesh.subscribe("BT_AI_key_press", self.BT_AI_key_press) + EventMesh.subscribe("stop_AI_record", self.stop_AI_record) + EventMesh.subscribe("AI_key_release", self.AI_key_release) + EventMesh.subscribe("pwk_key_double_press", self.pwk_key_double_press) + EventMesh.subscribe("AI_key_long_press", self.AI_key_long_press) + EventMesh.subscribe("pwk_key_release", self.pwk_key_release) + EventMesh.subscribe("get_AI_audio_queue", self.get_AI_audio_queue) + EventMesh.subscribe("get_VC_satrt_flag", self.get_VC_satrt_flag) + EventMesh.subscribe("get_vc_stream_active", self.get_vc_stream_active) + EventMesh.subscribe("get_vc_tts_playback_active", self.get_vc_tts_playback_active) + EventMesh.subscribe("battery_priority_mute_vc", self.battery_priority_mute_vc) + EventMesh.subscribe("VC_error", self.VC_error) + EventMesh.subscribe("audio_test", self.audio_test) + EventMesh.subscribe("audio_test_close", self.audio_test_close) + EventMesh.subscribe("set_record_data_save_flag", self.set_record_data_save_flag) + EventMesh.subscribe("set_VC_TTS_voice_type", self.set_VC_TTS_voice_type) + EventMesh.subscribe("SET_TTS_VOLUME", self.set_tts_volume) + EventMesh.subscribe("set_VC_stop_play_audio_flag", self.set_VC_stop_play_audio_flag) + EventMesh.subscribe("set_yuyinaudio_flag",self.set_yuyinaudio_flag) + EventMesh.subscribe("get_yuyinaudio_flag",self.get_yuyinaudio_flag) + EventMesh.subscribe("set_pwk_cnt_voice",self.set_pwk_cnt_voice) + EventMesh.subscribe("get_pwk_cnt_voice",self.get_pwk_cnt_voice) + EventMesh.subscribe("exit_voice_message_mode",self.exit_voice_message_mode) #超时未退出语音留言模式 + EventMesh.subscribe("come_meet_mode",self.come_meet_mode)#进入会议模式 + EventMesh.subscribe("exit_meet_mode",self.exit_meet_mode)#退出会议模式 + EventMesh.subscribe("come_trans_mode",self.come_trans_mode)#进入翻译模式 + EventMesh.subscribe("exit_trans_mode",self.exit_trans_mode)#退出翻译模式 + EventMesh.subscribe("set_trans_flag",self.set_trans_flag)#设置翻译状态 + EventMesh.subscribe("get_audio_play_state",self.get_audio_play_state) + EventMesh.subscribe("set_voicerupt_flag",self.set_voicerupt_flag)#按键中断触发留言播放 + EventMesh.subscribe("set_wait_voicerupt_flag",self.set_wait_voicerupt_flag)#按键中断触发未读留言播放 + EventMesh.subscribe("get_recv_voice",self.get_recv_voice)#获取是否收到留言 + EventMesh.subscribe("rate_change",self.rate_change)#直接修改语速 + EventMesh.subscribe("get_history_voice_flag",self.get_history_voice_flag)#获取查找历史留言状态 + EventMesh.subscribe("set_history_voice_flag",self.set_history_voice_flag)#修改查找历史留言状态 + EventMesh.subscribe("clean_unread_vocie",self.clean_unread_vocie)#修改查找历史留言状态 + EventMesh.subscribe("get_wait_voice_flag",self.get_wait_voice_flag)#获取60S等待留言播放定时器状态 + EventMesh.subscribe("set_wait_voice_flag",self.set_wait_voice_flag)#修改60S等待留言播放定时器状态 + EventMesh.subscribe("rate_move",self.rate_move)#NLP修改语速 + EventMesh.subscribe("find_rate",self.find_rate)#NLP查询语速 + EventMesh.subscribe("set_pwk_notice_flag",self.set_pwk_notice_flag)#设置未读状态 + # EventMesh.subscribe("get_voicerupt_flag",self.get_voicerupt_flag)#获取留言播放状态 + EventMesh.subscribe("chenggong_tone",self.chenggong_tone) + + pass + EventMesh.publish("set_led_state", {"ai_state": "idle"}) + self.dialog_attributes_provider = EventMesh.publish("get_storage_config","dialog_attributes_provider")#智能体类型配置 + if self.dialog_attributes_provider == None: + self.dialog_attributes_provider = "coze" + self.outbound_attributess_provider = EventMesh.publish("get_storage_config","outbound_attributess_provider")#tts服务选择 + if self.outbound_attributess_provider == None: + self.outbound_attributess_provider = "volcengine" + self.inbound_attributess_provider = EventMesh.publish("get_storage_config","inbound_attributess_provider")#tts服务选择 + if self.inbound_attributess_provider == None: + self.inbound_attributess_provider = "volcengine" + + self.bot_name = "柒先生" + try: + if "bot_name" in self.VC_TTS_voice_type[self.VC_TTS_voice_key]: + self.bot_name = self.VC_TTS_voice_type[self.VC_TTS_voice_key]["bot_name"] + else: + print("not bot_name!!!!!!!!!!!!!!!!!!!!!!") + except Exception as e: + usys.print_exception(e) + try: + bt_ota_flag=0 + try: + if "vc_config" in globals() and vc_config and vc_config.get_bool("BT_OTA_ENABLED", False): + bt_ota_flag = self.ai_OTA.bt_ota()#先检查蓝牙是否要升级 + except Exception: + pass + + cat1_ota_flag = self.ai_OTA.min_sys_ota() + if bt_ota_flag==1 or cat1_ota_flag == 1: + #if 0: + if self.ota_update_notice != "": + self.AI_TTS_str = self.ota_update_notice + else: + self.AI_TTS_str = "检测到有新版本需要升级啦！升级时可千万别断电或乱按哦，升级完成后设备会自动重启，重启就代表升级成功" + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + self.Close_flag = 1 + # if self.ai_OTA.min_sys_ota()==2: + # self.AI_TTS_str = "检测到有新版本需要升级啦！电量低于40%，无法保证升级正常执行，请充满电后重启设备再次升级！" + # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + else: + try: + self.ai_OTA.ota_result() + self.ai_OTA.pots_ICCID() + except Exception as e: + usys.print_exception(e) + #开机提示语，如果联网失败，直接提示信号弱 + + + #EventMesh.publish("get_storage_config",["powerRestart",1]) + powerRestart = EventMesh.publish("get_storage_config","powerRestart")#=1非正常重启 + if powerRestart == None: + powerRestart = 0 + if powerRestart == 1: + EventMesh.publish("set_storage_config",["powerRestart",0]) + + else: + pass + self.AI_TTS_str = self.power_on_tone.replace("bot_name", self.bot_name) + print("开机提示：{}".format(self.AI_TTS_str)) + try: + self._pending_tts_kind = "boot_prompt" + self._last_tts_kind = "boot_prompt" + self._skip_started_uplink_once = 1 + except Exception: + pass + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + + except Exception as e: + usys.print_exception(e) + def set_pwk_notice_flag(self,event,msg):#设置未读状态 + self.pwk_notice_flag = msg + def get_wait_voice_flag(self,event,msg): + return self.wait_voice_flag + def set_wait_voice_flag(self,event,msg): + # print("未读留言标记修改为:{}".format(msg)) + self.wait_voice_flag = msg + def get_history_voice_flag(self,event,msg): + return self.history_voice_flag + def set_history_voice_flag(self,event,msg): + # print("历史留言标记修改为:{}".format(msg)) + self.history_voice_flag = msg + def clean_unread_vocie(self,event,msg): + self.pwk_cnt_voice = 0 + self.pwk_notice_flag = 0 + EventMesh.publish("set_app_new_voice_flag",0) + def audio_test(self,event = None, msg = None): + self._uplink_request_start("audio_test") + self.VC_Play_Start() + def audio_test_close(self,event = None, msg = None): + self._uplink_request_stop("audio_test_close") + self.VC_Play_End() + if self.record_data_save_flag == 1: + if self.record_fd : + self.record_fd.close() + self.record_fd = None + def post_processor_after_instantiation(self): + pass + def get_VC_satrt_flag(self, event, msg): + return self.VC_satrt_flag + def check_vc_start(self): + print("check_vc_start:{} {} {} {}".format(self.VC_satrt_flag,self.VC_record_state_flag,self.AI_client_VC_isconnect,self.VC_Play_flag)) + if self.VC_satrt_flag == 0: + if self.VC_record_state_flag != 0 or self.AI_client_VC_isconnect == 1 or self.VC_Play_flag == 1: + self.Close() + def VC_30S_timer_cb(self, *args): # + EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_30S_timer_cb_}) + def VC_30S_timer_cb_(self): + self.get_config_timer_count += 1 + if self.get_config_timer_count >= 2*60*2:#2小时更新配置参数 + self.get_config_timer_count = 0 + if self.http_get_config_flag == 1: + try: + self.http_get_config()#获取/更新配置参数 + except Exception as e: + usys.print_exception(e) + self.check_vc_start()#30秒检查一下有没有关闭 + def ud_error(self,error_data): + response = None + if self.is_up_error == 1: + try: + # currentVersion = versions.Build_Time + # currentVersion = "7PIN-"+currentVersion[:10]+"-"+currentVersion[11:-3] + url = "http://{}/health?task_id={}".format(self.server_ip,error_data[0]+error_data[1]) + print(url) + response = request.get(url,timeout=5) + if response.status_code == 200: + + print("ud_error:ok") + else: + print("ud_error error!!!!") + for i in response.text: + print(i) + response.close() + except Exception as e: + usys.print_exception(e) + if response != None: + response.close() + def http_get_config(self, event=None, msg=None): + print("http_get_config") + + response = None + #http://180.184.206.142/bz-api/dev/config?deviceId=xxx + try: + url = (constants.DMP_BASE + "/dev/config/{}").format(modem.getDevImei()) + + + print(url) + response = request.get(url,timeout=5) + if response.status_code == 200: + data = response.json() + print("http_get_config:{}".format(data)) + + if data: + try: + if data["code"] == 20000:#有配置 + if data["data"] : + if "version" in data["data"]: + config_version = EventMesh.publish("get_storage_config","config_version")#音量配置 + if config_version != data["data"]["version"]: + config_version = data["data"]["version"] + EventMesh.publish("set_storage_config",["config_version",config_version]) + if "VC_TTS_voice_type" in data["data"]: + self.VC_TTS_voice_type = data["data"]["VC_TTS_voice_type"] + EventMesh.publish("set_storage_config",["VC_TTS_voice_type",self.VC_TTS_voice_type]) + print("配置VC_TTS_voice_type成功") + if "server_ip" in data["data"]: + self.server_ip = data["data"]["server_ip"] + EventMesh.publish("set_storage_config",["server_ip",self.server_ip]) + print("配置server_ip成功") + if "power_on_tone" in data["data"]: + self.power_on_tone = data["data"]["power_on_tone"] + EventMesh.publish("set_storage_config",["power_on_tone",self.power_on_tone]) + print("配置power_on_tone成功") + if "dialog_attributes_provider" in data["data"]: + self.dialog_attributes_provider = data["data"]["dialog_attributes_provider"] + EventMesh.publish("set_storage_config",["dialog_attributes_provider",self.dialog_attributes_provider]) + if "outbound_attributess_provider" in data["data"]: + self.outbound_attributess_provider = data["data"]["outbound_attributess_provider"] + EventMesh.publish("set_storage_config",["outbound_attributess_provider",self.outbound_attributess_provider]) + print("配置outbound_attributess_provider成功") + if "inbound_attributess_provider" in data["data"]: + self.inbound_attributess_provider = data["data"]["inbound_attributess_provider"] + EventMesh.publish("set_storage_config",["inbound_attributess_provider",self.inbound_attributess_provider]) + print("配置inbound_attributess_provider成功") + if "speech_rate" in data["data"]: + self.speech_rate = EventMesh.publish("SET_SPEECH_RATE",float(data["data"]["speech_rate"])) + EventMesh.publish("set_storage_config",["speech_rate",self.speech_rate]) + print("配置speech_rate成功") + if "emotion" in data["data"]: + self.emotion = data["data"]["emotion"] + EventMesh.publish("set_storage_config",["emotion",self.emotion]) + print("配置emotion成功") + if "ota_update_notice" in data["data"]: + self.ota_update_notice =data["data"]["ota_update_notice"] + EventMesh.publish("set_storage_config",["ota_update_notice",self.ota_update_notice]) + print("配置ota_update_notice成功") + if "VC_TTS_volume" in data["data"]: + self.VC_TTS_volume =int(data["data"]["VC_TTS_volume"]) + EventMesh.publish("set_storage_config",["VC_TTS_volume",self.VC_TTS_volume]) + print("配置VC_TTS_volume成功") + + except Exception as e: + usys.print_exception(e) + else: + print("status_code:{}".format(response.status_code)) + print("http_get_config:error!!!!!!!!!!") + response.close() + except Exception as e: + usys.print_exception(e) + if response != None: + response.close() + def get_yuyinaudio_flag(self, event=None, msg = None): + return self.voice_flag + def set_yuyinaudio_flag(self, event=None, msg = None): + self.voice_flag=0 + def get_pwk_cnt_voice(self, event=None, msg = None): + return self.pwk_cnt_voice + def set_pwk_cnt_voice(self, event=None, msg = None): + self.pwk_cnt_voice=msg + print("设置未读次数为：{}".format(self.pwk_cnt_voice)) + def get_meetaudio_flag(self, event=None, msg = None): + return self.meet_flag + def set_meetaudio_flag(self, event=None, msg = None): + self.meet_flag = 0 + def come_meet_mode(self, event=None, msg = None): + if self.meet_flag == 0: #还是为0就提示进入会议纪要模式 + EventMesh.publish("send_meet_open_flag_to_iot")#创建会议 + self.meet_flag=1 + exit_tts ="开启会议纪要，正在录音" + EventMesh.publish("play_tts",[exit_tts,1]) + utime.sleep_ms(4000) + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_key_press}) + pass + def exit_meet_mode(self, event=None, msg = None): + if self.meet_flag == 1: #还是为1就提示已退出会议纪要模式 + EventMesh.publish("send_meet_end_flag_to_iot")#结束会议 + self.meet_flag=0 + EventMesh.publish("stop_voice_record") #关闭语音录音 + exit_tts ="录音结束，已退出会议纪要模式" + EventMesh.publish("play_tts",[exit_tts,1]) + pass + def come_trans_mode(self, event=None, msg = None): + if self.trans_flag == 0: #还是为0就提示进入AI翻译模式 + EventMesh.publish("send_trans_come_flag_to_iot")#进入翻译 + pass + def exit_trans_mode(self, event=None, msg = None): + if self.trans_flag == 1: #还是为1就提示已退出AI翻译模式 + EventMesh.publish("stop_voice_record") #关闭语音录音 + exit_tts ="已退出AI翻译模式" + EventMesh.publish("play_tts",[exit_tts,1]) + self.trans_flag=0 + EventMesh.publish("send_trans_end_flag_to_iot")#退出翻译 + pass + def set_trans_flag(self,event,msg): + self.trans_flag = msg + def exit_voice_message_mode(self, event=None, msg = None): + if self.voice_flag == 1: #还是为1就提示已退出会议纪要模式 + self.voice_flag=0 + exit_tts ="已退出语音留言模式" + EventMesh.publish("play_tts",[exit_tts,1]) + pass + def ai_key_press_audio_cb1(self, event=None, msg = None): + print("播放按键音!!!!!!!!!!") + # EventMesh.publish("audio_montage_play",["chenggong.mp3"]) + # self.chenggong_tone() + self.aud.aud_tone_play(10,400)#播放按键音 + def ai_key_press_audio_cb2(self, event=None, msg = None): + EventMesh.publish("audio_montage_play",["chenggong.mp3"]) + def AI_key_press(self, event=None, msg = None):#按键按下 + # 按键时打印 LED 聚合状态/最终模式（用于定位“绿灯不灭/闪一下”） + try: + if "vc_config" in globals() and vc_config and vc_config.get_bool("KEY_LED_DEBUG", False): + try: + led = EventMesh.publish("get_led_debug_state") + except Exception: + led = None + try: + t = getattr(self.fsm, "turn", None) + except Exception: + t = None + print("[KEY] AI_key_press VC_mode={} {} turn={} led={}".format( + getattr(self, "VC_mode", -1), self._vc_fsm_log_snip(), t, led)) + except Exception: + pass + # 单对话 PTT / 长对话分支（按键短按提示音已关，避免与开录等提示音叠在一起） + try: + if self._ai_key_press_ptt_mode(): + return + except Exception as e: + usys.print_exception(e) + try: + if self._ai_key_press_long_dialogue_interrupt(): + return + except Exception as e: + usys.print_exception(e) + try: + if self._ai_key_press_long_dialogue_when_session_ready(): + return + except Exception as e: + usys.print_exception(e) + try: + if self._ai_key_press_long_dialogue_wake(): + EventMesh.publish(constants.EVENT.SYSTEM_UP) + return + except Exception as e: + usys.print_exception(e) + + EventMesh.publish(constants.EVENT.SYSTEM_UP)#唤醒休眠 + if EventMesh.publish("get_sos_flag")== 1: + return + if self.voice_flag == 1: #语音留言模式 + self.stop_voice_timer.stop() #关闭语音留言模式定时器 + # EventMesh.publish("stop_AI_record") + self.Close() + self.revoice_cnt = 0 + self.revoice_timer.stop()#回复定时器关闭，计数器清零 + EventMesh.publish("start_voice_record",2)#开始语音录音 + return + elif self.meet_flag == 1:#会议纪要模式 + self.Close() + EventMesh.publish("start_voice_record",3)#开始会议录音 + return + elif self.trans_flag == 1:#AI翻译模式 + return + + EventMesh.publish("stop_voice_record") #关闭语音录音 + if self.VC_mode == 0: + print("[KEY] AI_key_press fallback -> AI_VC_start ws={} started={}".format( + getattr(self, "AI_client_VC_isconnect", -1), + getattr(self, "_vc_session_started", -1))) + try: + self.Close_flag = 0 + except Exception: + pass + self.get_config_timer_count = 0 + self.key_state = 1 + self.AI_TTS_str = "" + self.AI_ASR_str = "" + self.AI_ASR_str_tmp = "" + EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_VC_start}) + self.AI_key_long_press_flag = 0 + return + print("AI_key_press: unhandled mode branch VC_mode={}".format( + getattr(self, "VC_mode", -1))) + self.AI_key_long_press_flag = 0 + def BT_AI_key_press(self, event=None, msg = None):#按键按下 + EventMesh.publish(constants.EVENT.SYSTEM_UP)#唤醒休眠 + if EventMesh.publish("get_sos_flag")== 1: + return + if self.voice_flag == 1: #语音留言模式 + tts_str = "语音留言模式，长按按钮开始录音，松开按钮结束" + EventMesh.publish("play_tts",[tts_str,1]) + return + + EventMesh.publish("stop_voice_record") #开启静默需要关闭 + print("AI_key_press") + self.get_config_timer_count = 0 + self.key_state = 1 + # EventMesh.publish("audio_montage_play",["chenggong.mp3"]) + # self.chenggong_tone() #播放按键音 + # 长对话：蓝牙按键同单按 + # - 对话进行中：打断/续聊（复用 dialog_id） + # - 已停止/未 started：走新会话流程 + try: + if self.VC_mode == 0: + if getattr(self, "AI_client_VC_isconnect", 0) == 1 and getattr(self, "_vc_session_started", 0) == 1: + EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_RequestToSpeak}) + return + except Exception: + pass + + self.Close() + + if self.VC_mode == 1: #长按对话模式 接收到蓝牙耳机按键事件 + + tts_str = "请您按住圆形的按钮开始讲话，讲完松开即可 " + EventMesh.publish("play_tts",[tts_str,1]) + return + # if dev_type == "7PIN": + # self.chenggong_tone() + #self.aud.aud_tone_play(10,200)#播放按键音 + self.AI_TTS_str = ""#"你好，有什么吩咐" + self.AI_ASR_str = "" + self.AI_ASR_str_tmp = "" + # if self.VC_state != VC_State_None: + # EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) + # EventMesh.publish("publish_AI_btnInfo",{"func":utime.sleep_ms,'data':(10)}) + self._uplink_request_stop("BT_AI_key_press/compat2") + self._uplink_request_start("BT_AI_key_press/compat2") + + self.AI_VC_start() + + + # self.Close() + # self.AI_VC_start() + self.AI_key_long_press_flag = 0 + def stop_AI_record(self, event=None, msg=None):#停止AI对话 + if self.VC_satrt_flag == 1: + self.get_config_timer_count = 0 + self.AI_TTS_str = ""#"你好，有什么吩咐" + self.AI_ASR_str = "" + self.AI_ASR_str_tmp = "" + self.AI_key_long_press_flag = 0 + EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) + self.request_uplink_stop("stop_AI_record") + count = 0 + while True: + if count >= 20 or self.VC_satrt_flag == 0: + break + count+=1 + utime.sleep_ms(500) + def exit_voice_message_timer(self,*argc): + EventMesh.publish("exit_voice_message_mode") + def AI_key_release(self, event=None, msg=None):#按键弹起 + try: + if "vc_config" in globals() and vc_config and vc_config.get_bool("KEY_LED_DEBUG", False): + try: + led = EventMesh.publish("get_led_debug_state") + except Exception: + led = None + try: + t = getattr(self.fsm, "turn", None) + except Exception: + t = None + print("[KEY] AI_key_release VC_mode={} {} turn={} led={}".format( + getattr(self, "VC_mode", -1), self._vc_fsm_log_snip(), t, led)) + except Exception: + pass + print("AI_key_release123") + self.key_state = 0 + if self.trans_flag == 1:#AI翻译模式 + EventMesh.publish("stop_voice_record") #关闭语音录音 + return + if self.voice_flag == 1: + self.stop_voice_timer.stop() + self.voice_flag=0 + if EventMesh.publish("GET_USB_STATE") == 0: + EventMesh.publish("set_led_state", {"ai_state": "idle"}) + EventMesh.publish("stop_voice_record")#停止静默录音 + tts_play_str = "留言已发送" + self.voice_flag=0 + EventMesh.publish("play_tts",[tts_play_str,1]) + return + if self.VC_mode == 1: + self._ai_key_release_handle_ptt() + return + def pwk_key_double_press(self,event,msg):#pwk双击 + print("come 双击") + # 切换模式：必须打断当前对话/录音/播放，回到 idle，等待按键触发新一轮 + try: + self.Close_flag = 1 + except Exception: + pass + try: + EventMesh.publish("publish_AI_btnInfo", {"func": self.Close}) + except Exception: + pass + if self.VC_mode == 0: + self.VC_mode = 1 + tts_str = "已切换到单对话模式" + # 下一次“无 dialog_id 的提示音 TTS”归类为模式切换提示（单对话） + try: + self._pending_tts_kind = "mode_switch_ptt" + self._last_tts_kind = "mode_switch_ptt" + except Exception: + pass + else: + self.VC_mode = 0 + tts_str = "已切换到长对话模式" + # 下一次“无 dialog_id 的提示音 TTS”归类为模式切换提示（长对话） + try: + self._pending_tts_kind = "mode_switch_long" + self._last_tts_kind = "mode_switch_long" + except Exception: + pass + # 提示音播完即退出（Close_flag=1），保持 idle，等待按键 + EventMesh.publish("play_tts",[tts_str,1]) + EventMesh.publish("set_storage_config",["VC_mode",self.VC_mode]) + def AI_key_long_press(self, event, msg):#按键长按 + try: + if "vc_config" in globals() and vc_config and vc_config.get_bool("KEY_LED_DEBUG", False): + try: + led = EventMesh.publish("get_led_debug_state") + except Exception: + led = None + try: + t = getattr(self.fsm, "turn", None) + except Exception: + t = None + print("[KEY] AI_key_long_press VC_mode={} {} turn={} led={}".format( + getattr(self, "VC_mode", -1), self._vc_fsm_log_snip(), t, led)) + except Exception: + pass + print("AI_key_long_press") + self._ai_key_long_press_route(event, msg) + def pwk_key_release(self, event, msg):#电源键短按，关闭 + try: + if "vc_config" in globals() and vc_config: + _m = bool(vc_config.get_bool("SINGLE_POWER_KEY_AI_MERGED", False)) + if not _m: + _ai = vc_config.get_attr("EXTINT_AI_GPIO", "GPIO22") + _m = _ai is None or _ai == "" + else: + _m = False + except Exception: + _m = False + if _m: + self._handle_pwk_short_release(event, msg, merged_ai_tail=True) + return + self._handle_pwk_short_release(event, msg, merged_ai_tail=False) + def key_prompt(self): + EventMesh.publish("audio_montage_play",["key_prompt.mp3"]) + def VC_error(self, event, msg):#错误播报 + self.websocket_error() + def set_voicerupt_flag(self,event,msg):#按键控制当前留言播放 + self.voicerupt_flag = msg + def set_wait_voicerupt_flag(self,event,msg):#按键控制未读留言播放 + self.wait_voicerupt_flag = msg + def get_voicerupt_flag(self,event,msg):#获取按键控制留言播放状态 + return self.voicerupt_flag + def _1S_timer_cb(self,*argc): + EventMesh.publish("publish_AI_btnInfo",{"func":self._1S_timer_cb_}) + def _fsm_timeout_table_tick(self, now_ms): + # 兜底统一入口：按“状态机状态 + 超时阈值 + 节流”驱动，避免兜底抢跑/多入口打架。 + # 只在对应 turn/session 状态触发动作。 + try: + turn = getattr(self.fsm, "turn", VCTurnState.Idle) + except Exception: + turn = VCTurnState.Idle + + # 1) synthesis→tts.started 超时：只在 Thinking 期允许清 pending 并重启会话 + # 根因：服务端未返回 tts.started 时，继续“回到聆听/开麦”会把本轮状态机拉扯得更乱； + # 这里直接重启会话，让协议重新进入 started→下一轮流程。 + try: + if turn == VCTurnState.Thinking and self._tts_pending == 1 and self._tts_pending_ms: + try: + if self._fb_last_tts_pending_timeout_ms: + tts_gap_ok = utime.ticks_diff(now_ms, self._fb_last_tts_pending_timeout_ms) > 15000 + else: + tts_gap_ok = True + except Exception: + tts_gap_ok = True + if utime.ticks_diff(now_ms, self._tts_pending_ms) > 8000 and tts_gap_ok: + self._fb_last_tts_pending_timeout_ms = now_ms + print("[VC][tts_pending] timeout, clear pending and restart session") + self._tts_pending = 0 + self._tts_pending_ms = 0 + if self.VC_mode == 0 and self.Close_flag == 0: + try: + self.fsm.reset_rx_phases() + except Exception: + pass + EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_VC_start}) + except Exception: + pass + + # 2) thinking 超时：只在「llm.completed 已收到但 tts.started 长时间未到」时才重启会话 + # 注意：不能以 llm.created 的时间戳作为超时基准，否则会在 LLM 正常推理期间误判超时（notts 场景）。 + try: + if self.VC_mode == 0 and self.Close_flag == 0: + llm_completed_ms = getattr(self, "_llm_completed_ms", 0) + # 只有在 llm.completed 之后才允许超时重启 + if turn == VCTurnState.Thinking and llm_completed_ms and self._turn_has_tts_started == 0: + age_ms = utime.ticks_diff(now_ms, llm_completed_ms) + restart_threshold_ms = 20000 + # _fb_last_thinking_restart_ms==0 时 ticks_diff(now,0) 非预期（间隔被误判极大）， + # 会在 llm.completed 后首批 tick 就触发 AI_VC_start → notts（无 tts.started）；首次仅信任 age 门限 + try: + if self._fb_last_thinking_restart_ms: + gap_ok = utime.ticks_diff(now_ms, self._fb_last_thinking_restart_ms) > 15000 + else: + gap_ok = True + except Exception: + gap_ok = True + if (age_ms > restart_threshold_ms and gap_ok and + self._tts_pending == 0 and self._need_audio_played_ack == 0 and self.VC_Play_flag == 0): + self._fb_last_thinking_restart_ms = now_ms + self._llm_completed_ms = 0 + self._server_vad_silence = 0 + try: + self.fsm.reset_rx_phases() + except Exception: + pass + try: + print("[VC][thinking_timeout] AI_VC_start age_ms={} thr={}".format( + age_ms, restart_threshold_ms)) + except Exception: + pass + self._vc_led_alert_rb_flash() + # thinking_timeout 策略：重启会话（而不是抢开麦），避免干扰服务端 LLM/TTS 流程 + EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_VC_start}) + except Exception: + pass + def _1S_timer_cb_(self): + try: + self._vc_tick_serial_cancel_timeout() + except Exception: + pass + + EventMesh.publish("system_sleep_rest")#防止黑屏 + # --- Level 1：仅长对话(VC_mode==0)，无业务下行 idle_sec 后结束会话（需再次按键进入） + # close_timerout：保活清除策略见 _vc_rx_resets_long_idle_timer（排除文本 ping/pong、heartbeat 等） + # 与浅休眠/灭屏解耦；单对话不走本计时 + idle_sec = 20 + try: + if "vc_config" in globals() and vc_config: + idle_sec = int(vc_config.get_int("VC_IDLE_AUTO_CLOSE_SEC", 20)) + except Exception: + idle_sec = 20 + if idle_sec < 5: + idle_sec = 5 + if self.VC_mode == 0 and self.VC_satrt_flag == 1 and getattr(self, "AI_client_VC_isconnect", 0) == 1: + self.close_timerout += 1 + else: + self.close_timerout = 0 + if self.VC_mode == 0 and idle_sec > 0 and self.close_timerout > idle_sec: + try: + print("[VC][idle_L1] 长对话无业务下行超时: elapsed={}s thr={}s VC_mode={} VC_satrt={} ws_conn={} {}".format( + self.close_timerout, idle_sec, self.VC_mode, + getattr(self, "VC_satrt_flag", -1), getattr(self, "AI_client_VC_isconnect", -1), + self._vc_fsm_log_snip())) + except Exception: + pass + if self._vc_long_idle_active_speech_or_later(): + try: + print("[VC][idle_L1] 处理=websocket_error（对话中无服务端响应）") + except Exception: + pass + print("[VC][long_idle] branch=speech_no_rsp -> websocket_error + ud_error") + print("接口无响应，超时退出") + EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_error,"data":"接口超时了。请重试"}) + self.ud_error(["VC",self.VC_taskid]) + else: + try: + print("[VC][idle_L1] 处理=Close（结束对话，需再次按键进入）") + except Exception: + pass + print("[VC][long_idle] branch=idle_timeout -> Close") + EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) + #EventMesh.publish("sleep_BT")#休眠蓝牙 + + # --- Level 2：长对话+单对话，业务 idle 累计 VC_IDLE_DEEP_SLEEP_SEC → 断开（若仍连）+ 深睡事件 + deep_sec = 60 + deep_en = 1 + try: + if "vc_config" in globals() and vc_config: + deep_sec = int(vc_config.get_int("VC_IDLE_DEEP_SLEEP_SEC", 60)) + deep_en = int(vc_config.get_int("VC_IDLE_DEEP_SLEEP_ENABLE", 1)) + except Exception: + deep_sec, deep_en = 60, 1 + active_deep = bool(deep_en) and deep_sec > 0 + if active_deep and deep_sec < 10: + deep_sec = 10 + if not active_deep: + self._idle_deep_sec = 0 + elif not self._vc_is_business_idle(): + self._idle_deep_sec = 0 + try: + self._vc_idle_deep_armed_suppress = 0 + except Exception: + pass + elif getattr(self, "_vc_idle_deep_armed_suppress", 0): + pass + else: + self._idle_deep_sec += 1 + if active_deep and self._idle_deep_sec > deep_sec: + try: + print("[VC][idle_L2] 业务idle累计超时: elapsed={}s thr={}s VC_mode={} VC_satrt={} ws_conn={}".format( + self._idle_deep_sec, deep_sec, self.VC_mode, + getattr(self, "VC_satrt_flag", -1), getattr(self, "AI_client_VC_isconnect", -1))) + except Exception: + pass + try: + self._vc_idle_deep_armed_suppress = 1 + except Exception: + pass + self._idle_deep_sec = 0 + EventMesh.publish("publish_AI_btnInfo", {"func": self._vc_idle_deep_sleep_impl}) + + # 统一兜底表（按状态机状态触发） + try: + self._fsm_timeout_table_tick(utime.ticks_ms()) + except Exception: + pass + + # （不做额外兜底）动态切换依赖 FSM 状态正确清理后，由 server.tts.completed→local.play_end 驱动自动开麦 + + # print("get_audio_state !!!!!{} {} {}\r\n".format(self.play_end,EventMesh.publish("get_audio_state"),self.play_flag)) + # ===== 播放结束检测：用 FSM 驱动 play_end 动作链 ===== + try: + self._vc_try_play_end_tick() + except Exception: + pass + if self.online_kws == 1: + if self.VC_satrt_kws_flag == 1 and self.VC_satrt_flag == 0:#要打开录音上传数据实现kws + pass + + # thinking/llm→tts 等兜底已在 _fsm_timeout_table_tick 内统一处理 + # VC WebSocket 文本 ping：间隔由 default_config 的 VC_WS_PING_INTERVAL_SEC 控制（秒），<=0 表示关闭 ping + try: + _ping_iv = int(vc_config.get_int("VC_WS_PING_INTERVAL_SEC", 4)) + except Exception: + _ping_iv = 4 + if _ping_iv > 0: + self.VC_pingping_timerout += 1 + if self.VC_pingping_timerout >= _ping_iv: + self.VC_pingping_timerout = 0 + self.AI_client_VC_ping() + # self.asr_pingping_timerout += 1 + # if self.asr_pingping_timerout>3: + # self.asr_pingping_timerout=0 + # self.AI_client_asr_ping() + # 历史遗留：VC_Thinking_timerout 的重启通道会在 llm.created 后抢跑触发 AI_VC_start， + # 直接打断本轮 LLM→TTS（notts 日志：llm.created 后立刻 AI_VC_start，导致无 tts.started）。 + # 现在统一由 _fsm_timeout_table_tick(Thinking 超时) 负责重启会话，因此禁用该旧通道。 + if self.VC_Thinking_timerout > 0 and self.VC_Thinking_timerout < 100: + self.VC_Thinking_timerout = 100 + def AI_VC_start(self): + print("AI_VC_start1") + + EventMesh.publish(constants.EVENT.SYSTEM_UP)#唤醒休眠 + self.VC_satrt_flag = 1 + self.VC_stop_play_audio_flag = 1 + self.VC_satrt_kws_flag = 1 + if self.VC_Thinking_timerout < 100: + self.VC_Thinking_timerout = 0 + # 不在这里提前切灯态：播放/聆听灯只在“播放/录音真正开始/停止”处更新 + EventMesh.publish("wakeup_BT") + #self.VC_Play_End()#关闭音频播放 + print("AI_VC_start2") + #self.aud.aud_tone_play(16,200)#播放按键音 + self.LocalRespondingEnded_timerout = 0 + self.close_timerout = 0 + self._idle_deep_sec = 0 + self._vc_idle_deep_armed_suppress = 0 + + # + if self.AI_client_VC_isconnect == 0:#没连接的时候请求开始 + self.websocket_connect_VC()#连接服务器 + + #utime.sleep_ms(50) + else: + print("AI_VC_start2 (already connected) {}".format(self._vc_fsm_log_snip())) + EventMesh.publish(constants.EVENT.SYSTEM_UP)#唤醒休眠 + if self.AI_client_VC_isconnect == 0: + #self.Close() + return + # 单对话 PTT：会话已 started 时勿重复发 Start（否则第二轮起上行/ASR 易乱） + _ptt_skip_start = False + try: + if self.VC_mode == 1 and getattr(self, "_vc_session_started", 0) == 1: + _ptt_skip_start = True + print("[PTT] AI_VC_start skip duplicate Start (session active)") + except Exception: + pass + if not _ptt_skip_start: + self.AI_client_VC_start_requset() + print("AI_VC_start3") + + if (not _ptt_skip_start) and self.AI_TTS_str == "" and self.AI_new_voice_url == "": + self._uplink_request_start("AI_VC_start") + + + # pass + print("AI_VC_start4") + + #self.VC_Play_Start() + #EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Play_Start}) + #开始线程 + + # if self.AI_VC_send_data_queue_thread_id == 0: + # _thread.stack_size(self.Audio_buf_size+8*1024) + # self.AI_VC_send_data_queue_thread_id = _thread.start_new_thread(self.AI_VC_send_data_queue_thread, ()) + # _thread.stack_size(8*1024) + # print("AI_VC_start :AI_VC_send_data_queue_thread_id :{} ".format(self.AI_VC_send_data_queue_thread_id)) + + if self.AI_audio_queue_thread_id ==0: + _thread.stack_size(24*1024) + self.AI_audio_queue_thread_id = _thread.start_new_thread(self.AI_audio_queue_thread, ()) + _thread.stack_size(8*1024) + print("AI_VC_start :AI_audio_queue_thread_id :{} ".format(self.AI_audio_queue_thread_id)) + else: + print("AI_VC_start :AI_audio_queue_thread_id != 0 ") + EventMesh.publish(constants.EVENT.SYSTEM_UP)#唤醒休眠 + self._1S_timer.start(1000,1,self._1S_timer_cb) + # self.VC_pingping_timerout = 0 + # self.asr_pingping_timerout = 0 + print("AI_VC_start5") + def kws_even(self,event = None,msg= None): + EventMesh.publish("publish_AI_btnInfo",{"func":self.kws_even_cb}) + def kws_even_cb(self): + if EventMesh.publish("get_sos_flag") == 1: + return + print("kws_even_cb") + if self.VC_mode == 1: + try: + tts_str = vc_config.get_attr("KWS_PTT_HINT", "请您按住圆形的按钮开始讲话，讲完松开即可 ") + except Exception: + tts_str = "请您按住圆形的按钮开始讲话，讲完松开即可 " + EventMesh.publish("play_tts",[tts_str,1]) + return + if self.VC_mode == 0: + if EventMesh.publish("get_voice_record_flag") == 1:#停止静默录音 + if EventMesh.publish("get_voice_record_mode") == 2:#语音留言不能被打断 + return + elif EventMesh.publish("get_voice_record_mode") == 1: #静默录音需要打断 + EventMesh.publish("stop_voice_record")#停止静默录音 + self.aud.aud_tone_play(10,400) + # EventMesh.publish("audio_montage_play",["chenggong.mp3"]) + self.Close() + # if dev_type == "7PIN" : + # self.aud.aud_tone_play(10,400)#播放按键音 + #self.aud.aud_tone_play(10,200)#播放按键音 + self.AI_TTS_str = ""#"你好，有什么吩咐" + self.AI_ASR_str = "" + self.AI_ASR_str_tmp = "" + self._uplink_request_stop("kws_even_cb") + self._uplink_request_start("kws_even_cb") + self.AI_VC_start() + def set_wait_first_flag(self,event = None,msg= None):#设置未读语音播报初次记录flag + self.wait_first_flag = msg + def find_cmd(self,event ,msg): + if "电量" in msg: + battery = EventMesh.publish("get_battery") + print("电量查询：{}".format(battery)) + tts_play_str = "电量剩余百分之{}".format(battery) + EventMesh.publish("play_tts",[tts_play_str,0]) + # print("电量！！！！！！！！！！！") + elif "信号" in msg: + signal = net.csqQueryPoll() + tts_play_str = "当前信号{}".format(signal) + EventMesh.publish("play_tts",[tts_play_str,0]) + # print("信号！！！！！！！！！！！") + elif "卡号" in msg: + # print("卡号！！！！！！！！！！！") + iccid = sim.getIccid() + iccid = num_str_to_chinese(iccid)#转为中文 + if iccid != -1 and len(iccid) > 5: + print("iccid{}".format(iccid)) + tts_play_str = "当前卡号为:{}".format(iccid) + else: + tts_play_str = "卡号查询失败" + EventMesh.publish("play_tts",[tts_play_str,0]) + elif "设备号" in msg: + imei = modem.getDevImei() + if imei != -1 and len(imei) > 5: + print("IMEI：{}".format(imei)) + tts_play_str = "当前设备号为:{}".format(imei) + else: + tts_play_str = "设备号查询失败" + EventMesh.publish("play_tts",[tts_play_str,0]) + elif "音量" in msg : + tts_play_str = "当前音量是{}".format(self.VC_TTS_volume) + EventMesh.publish("play_tts",[tts_play_str,0]) + elif "语速" in msg : + tts_play_str = "当前语速是{}".format(EventMesh.publish("find_rate")) + EventMesh.publish("play_tts",[tts_play_str,0]) + def chenggong_tone(self): + EventMesh.publish("audio_montage_play",["chenggong.mp3"]) + def concurrency_limit_exceeded_tone(self): + self.Close() + EventMesh.publish("audio_montage_play",["chaobingfatis.mp3"]) #播放超并发提醒 + def factory_mode(self): + print("factory_mode") + EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) + EventMesh.publish("publish_AI_btnInfo",{"func":self.factory_start}) + def factory_start(self): + self.factory_timer.start(3000, 1, self.factory_timer_cb) + def factory_timer_cb(self, *args, **kwargs): + print("factory_timer_cb play......") + #self.aud.play(1,0,'U:/static/peizhichenggong.mp3')#切换成功 + EventMesh.publish("audio_montage_play",["xinhaoruo1.mp3"]) + def Close_VC(self,event,msg): + EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) + def Close(self): + try: + if getattr(self, "VC_mode", 0) != 1: + print("[VC][long_idle] Close() enter VC_mode={} {} VC_satrt_flag={} VC_record_state_flag={}".format( + getattr(self, "VC_mode", -1), self._vc_fsm_log_snip(), + getattr(self, "VC_satrt_flag", -1), getattr(self, "VC_record_state_flag", -1))) + except Exception: + pass + self.VC_stop_play_audio_flag = 0 + try: + self.AI_key_long_press_flag = 0 + except Exception: + pass + self.VC_Thinking_timerout = 0 + self.AI_ASR_timer.stop() + self._1S_timer.stop() + self.close_timerout = 0 + self._idle_deep_sec = 0 + self.factory_timer.stop() + self.AI_TTS_str = "" + self.AI_ASR_str = "" + self.AI_ASR_str_tmp = "" + self.AI_ASR_flag = 0 + self.AI_ASR_check_timeout = 0 + self.AI_ASR_error_timeout = 0 + self.LocalRespondingEnded_timerout = 0 + self._need_audio_played_ack = 0 + self._tts_pending = 0 + self._tts_pending_ms = 0 + self._vc_session_started = 0 + self._ptt_audio_created = 0 + self._vc_pending_interrupt_uplink = 0 + try: + self._skip_started_uplink_once = 0 + except Exception: + pass + try: + self._ptt_abort_interrupt_arm() + except Exception: + pass + # Power 短按停止对话后，再次按 AI 需要新建 dialog_id：清理本轮 dialog/session 标识 + try: + self._cur_dialog_id = "" + except Exception: + pass + try: + self._cur_sid = "" + except Exception: + pass + self.is_ruppt_tts = 0#打断标志位清零 + print("ai_VC Close1!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") + try: + if getattr(self, "VC_mode", 0) != 1: + print("[VC][long_idle] step=set_led_state_idle (first)") + except Exception: + pass + EventMesh.publish("set_led_state", {"ai_state": "idle"}) + self.tts_in_data_len = 0 + self.tts_out_data_len = 0 + #必须先销毁线程，不然线程会容易报错 + + try: + self.AI_audio_queue_thread_close()#销毁音频线程 + except Exception as e: + usys.print_exception(e) + try: + self.AI_VC_send_data_queue_thread_close()#销毁数据发送线程 + except Exception as e: + usys.print_exception(e) + + # Close 可能被不同线程触发：统一切回主线程执行 stop + try: + if getattr(self, "VC_mode", 0) != 1: + print("[VC][long_idle] step=schedule_uplink_stop reason=Close (async publish_AI_btnInfo)") + except Exception: + pass + EventMesh.publish("publish_AI_btnInfo",{"func":self._uplink_request_stop, "data":"Close"}) + + print("ai_VC Close1.1!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") + + print("ai_VC Close1.2!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") + + try: + if getattr(self, "VC_mode", 0) != 1: + print("[VC][long_idle] step=websocket_deconnect_VC") + except Exception: + pass + try: + self.websocket_deconnect_VC() + except Exception as e: + usys.print_exception(e) + + + + + + print("ai_VC Close1.3!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") + + print("ai_VC Close2!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") + + + + + if self.record_data_save_flag == 1: + if self.record_fd : + self.record_fd.close() + self.record_fd = None + if self.play_data_save_flag == 1: + if self.play_fd !=None: + self.play_fd.close() + self.play_fd = None + print("ai_VC Close3!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\r\n") + + if self.AI_VC_send_data_queue.empty()!=True: + self.AI_VC_send_data_queue.clear() + self.play_new_voice_flag = 0 #语音消息播放状态 + if self.voice_choose_flag == 0: # 如果没有未读语音 + self.AI_new_voice_url = "" # 语音播放地址 + print("关闭并重置语音标志位") + else: + print("关闭但是没有重置语音标志位") + self.AI_ASR_send_buf_len = 0 + #self.AI_WS_connect_Queue.put(self.AI_client_VC) + try: + if getattr(self, "VC_mode", 0) != 1: + print("[VC][long_idle] step=set_led_state_idle (second) -> VC_init VC_satrt_flag=0") + except Exception: + pass + EventMesh.publish("set_led_state", {"ai_state": "idle"}) + self.VC_init() + self.VC_satrt_flag = 0 + # Level2：会话结束后仍可继续 1s 节拍累计 idle 直至深睡（依赖 VC_IDLE_DEEP_SLEEP_SEC） + try: + _dsec = 60 + _den = 1 + if "vc_config" in globals() and vc_config: + _dsec = int(vc_config.get_int("VC_IDLE_DEEP_SLEEP_SEC", 60)) + _den = int(vc_config.get_int("VC_IDLE_DEEP_SLEEP_ENABLE", 1)) + if _den and _dsec > 0: + self._1S_timer.start(1000, 1, self._1S_timer_cb) + except Exception: + pass + def VC_init2(self): + self.VC_init() + + print("ai_VC VC_init2!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") + try: + self.websocket_deconnect_VC() + except Exception as e: + usys.print_exception(e) + print("ai_VC Close3!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\r\n") + self._1S_timer.stop() + if self.AI_VC_send_data_queue.empty()!=True: + self.AI_VC_send_data_queue.clear() + + self.AI_ASR_send_buf_len = 0 + if self.AI_audio_queue.empty()!=True: + self.AI_audio_queue.clear() + def VC_init(self): + _q0 = self.AI_audio_queue.size() + self.VC_Play_End() + if self.AI_audio_queue.empty()!=True: + self.AI_audio_queue.clear() + print("[VC] init audio_queue {} -> {}".format(_q0, self.AI_audio_queue.size())) diff --git a/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_session_imports.py b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_session_imports.py new file mode 100644 index 00000000..7a970606 --- /dev/null +++ b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_session_imports.py @@ -0,0 +1,77 @@ +"""vc_session 子模块共享 import（io / media / core 显式引用，禁止 import * 扩散到业务外）。""" +# -*- coding: utf-8 -*- +"""vc_session 子模块共享 import（io / media / core 显式引用，禁止 import * 扩散到业务外）。""" +from common import Abstract +try: + from qp_log import qp_print as print +except Exception: + pass +import EventMesh +import uwebsocket +import usys +import gc +import urandom +import uhashlib +import ubinascii +import ujson +import osTimer +import _thread +from queue import Queue +import utime +import audio +import uos +from machine import Pin +import modem +import sim +import net +import base64 +import request +import ql_fs +from misc import Power +import ntptime +import machine +import dataCall +import constants +import versions +import version_util +try: + import vc_config +except Exception: + vc_config = None +try: + import vc_text_intent +except Exception: + vc_text_intent = None +try: + import VAD +except Exception: + pass +try: + import usr.ai_alarm as _ai_alarm +except Exception: + import ai_alarm as _ai_alarm +from vc_common import ( + dev_type, + VC_HEARTBEAT_LOG, + VC_PROTO_GUARDS, + VC_WS_IO_LOG, + _VC_KEEPALIVE_RX_EVENTS, + _vc_out_json_event_meta, + PRESS_AUDIO_MODE, +) +from vc_fsm import ( + VCSessionState, + VCTurnState, + VC_TtsRxPhase, + VCMode, + VCStateMachine, + VC_record_TYPE_PCM, + VC_record_TYPE_AMR, + VC_play_TYPE_PCM, + VC_play_TYPE_MP3, + AI_client_VC_num, + AI_client_asr_num, +) +from vc_ota import OTA +from vc_utils import num_str_to_chinese + diff --git a/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_session_io.py b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_session_io.py new file mode 100644 index 00000000..185997d7 --- /dev/null +++ b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_session_io.py @@ -0,0 +1,1740 @@ +# -*- coding: utf-8 -*- +"""AI_VC mixin：VC WebSocket 建链、收发线程、协议 JSON 分发。""" +from vc_session_imports import * # noqa: F401,F403 +# import * 不导入以下划线开头的符号；_thread 必须显式引入（否则 start_new_thread NameError） +import _thread + +class VCSessionIoMixin: + def _vc_fsm_dispatch_cb(self, data): + """btn 线程：处理服务端 FSM 事件（单对话串行）。""" + try: + ev, jsload = data + except Exception: + return + try: + actions = self.fsm.on_server_event(ev, jsload) + self._fsm_apply_actions(actions) + except Exception as e: + usys.print_exception(e) + if ev == "canceled": + self._vc_on_ptt_cancel_settled(via="canceled") + def _vc_dispatch_server_event(self, ev, jsload_data=None): + try: + if ev == "canceled": + self._vc_stop_downlink_for_interrupt("canceled") + except Exception: + pass + if getattr(self, "VC_mode", -1) == 1: + self._vc_enqueue_btn(self._vc_fsm_dispatch_cb, (ev, jsload_data)) + else: + try: + self._fsm_apply_actions(self.fsm.on_server_event(ev, jsload_data)) + except Exception: + pass + def request_send_start(self): + EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_start_requset}) + def request_send_audio_create(self): + EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_send_speech}) + def request_send_audio_completed(self): + EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_end_speech}) + def request_send_audio_played(self): + EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_audio_played}) + def request_send_synthesis(self, text): + try: + self.AI_TTS_str = text + except Exception: + pass + EventMesh.publish("publish_AI_btnInfo", {"func": self.AI_client_VC_synthesis}) + def websocket_connect_VC(self): + print("websocket_connect_VC:{} e:{}!!!!!!!!!!!!!!!!!!!!!".format(self.AI_client_VC_isconnect,self.AI_client_VC_debug)) + if self.AI_client_VC_isconnect == 1: + return + # 创建VC客户端 + try: + signal = net.csqQueryPoll() + if signal == -1 or sim.getStatus() != 1:#无蜂窝/无卡等：弱网提示 + self.websocket_error(play_weak_signal_tone=True) + return + + req_str = "" + ack = "" + if self.is_VC_ACK == 1: + ack = "ack=1" + lang = "lang=zh-cn"#普通话测试环境 + try: + if "lang" in self.VC_TTS_voice_type[self.VC_TTS_voice_key]: + lang = self.VC_TTS_voice_type[self.VC_TTS_voice_key]["lang"] + else: + print("not lang!!!!!!!!!!!!!!!!!!!!!!") + except Exception as e: + usys.print_exception(e) + + #女，男，童，粤，川 + self.bot_name = "柒先生" + try: + if "bot_name" in self.VC_TTS_voice_type[self.VC_TTS_voice_key]: + self.bot_name = self.VC_TTS_voice_type[self.VC_TTS_voice_key]["bot_name"] + else: + print("not bot_name!!!!!!!!!!!!!!!!!!!!!!") + except Exception as e: + usys.print_exception(e) + + # Headers = {'X-NLS-Token':'0c8b8f05a46f423096be64a9c4c46310'} + # self.AI_client_VC = uwebsocket.Client.connect('wss://nls-gateway-cn-beijing.aliyuncs.com/ws/v1',headers=Headers, debug=False) + token = EventMesh.publish("get_dev_token") + print("token:{}".format(token)) + + req_str = "vc/chat?is_auth={}".format(self.is_auth) + try: + if "vc_config" in globals() and vc_config: + base = vc_config.get_attr("VC_WS_PATH", "vc/chat") or "vc/chat" + req_str = "{}?is_auth={}".format(base, self.is_auth) + except Exception: + pass + if(token): + url = 'ws://{}/{}&dt={}'.format(self.server_ip,req_str,token) + else: + url = 'ws://{}/{}'.format(self.server_ip,req_str) + #url = 'ws://115.190.94.208:22080/vc/coze?dt={}'.format(token) + + if VC_WS_IO_LOG: + print("[VC] ws url {}".format(url)) + self.AI_client_VC = uwebsocket.Client.connect(url, debug=False) + #self.AI_WS_connect_Queue.put(self.AI_client_VC) + + + print("[VC] websocket_connect ok") + except Exception as e: #网络连接异常（已通过 CSQ/SIM，多为 URL/握手/服务端问题，勿播「信号弱」） + usys.print_exception(e) + print("websocket_connect_VC error!!!\r\n") + try: + print("[VC] ws connect exc: {}".format(e)) + except Exception: + pass + str_error = str(e) + if str_error.find("[Errno 22] EINVAL")!= -1: + print("[VC] websocket EINVAL: {}".format(str_error)) + self.websocket_error(play_weak_signal_tone=False) + EventMesh.publish("set_storage_config",["powerRestart",1])#调试临时关闭 + utime.sleep_ms(1000) + self.aud.stopPlayStream() + self.aud.stopAll() + Power.powerRestart() + + self.websocket_error(play_weak_signal_tone=False) + return + #print("AI_client_VC:{}".format(self.AI_client_VC)) + if self.AI_client_VC == None: + EventMesh.publish("update_dev_token")#获取token + self.websocket_error(play_weak_signal_tone=False) + return + self.AI_client_VC_debug+=1 + utime.sleep_ms(10) + self.AI_client_VC.settimeout(5) + try: + if self.AI_VC_send_data_queue_thread_id == 0: + _thread.stack_size(self.Audio_buf_size+8*1024) + self.AI_VC_send_data_queue_thread_id = _thread.start_new_thread( + self.AI_VC_send_data_queue_thread, ()) + _thread.stack_size(8*1024) + print("AI_VC_start :AI_VC_send_data_queue_thread_id :{} ".format( + self.AI_VC_send_data_queue_thread_id)) + self.AI_client_VC_recv_thread_close() + _thread.stack_size(self.Audio_buf_size+8*1024) + self.AI_client_VC_recv_id = _thread.start_new_thread( + self.AI_client_VC_recv, (self.AI_client_VC,)) + _thread.stack_size(8*1024) + if not self.AI_client_VC_recv_id: + raise OSError("VC recv thread start failed") + self.AI_client_VC_isconnect = 1 + print("websocket_connect_VC end") + except Exception as e: + usys.print_exception(e) + try: + print("[VC] ws thread start failed: {}".format(e)) + except Exception: + pass + self.AI_client_VC_isconnect = 0 + try: + self.websocket_deconnect_VC() + except Exception: + pass + def websocket_error(self, TTS_str=None, play_weak_signal_tone=False): + """play_weak_signal_tone=True 时播 xinhaoruo1（弱网/无卡类）；连接/协议失败请传 False，避免误报「信号弱」。""" + try: + if getattr(self, "VC_mode", 0) != 1: + print("[VC][long_idle] websocket_error enter VC_mode={} TTS_str={} weak_hint={} -> Close then uplink_stop(async)".format( + getattr(self, "VC_mode", -1), TTS_str, play_weak_signal_tone)) + except Exception: + pass + print("websocket_error1!!!!") + utime.sleep_ms(800) + self.Close() + print("websocket_error1.1!!!!") + print("websocket_error!!!!") + print("websocket_error!!!!") + self.VC_Thinking_timerout = 0 + self.VC_satrt_flag = 0 + self.AI_TTS_str = "" + self.AI_ASR_str = "" + self.AI_ASR_str_tmp = "" + self.AI_ASR_flag = 0 + # 可能由收包线程/其它线程触发：统一切回主线程执行 stop，避免“异常→重试→再异常”的循环 + try: + if getattr(self, "VC_mode", 0) != 1: + print("[VC][long_idle] websocket_error schedule uplink_stop reason=websocket_error + set_led_idle") + except Exception: + pass + EventMesh.publish("publish_AI_btnInfo",{"func":self._uplink_request_stop, "data":"websocket_error"}) + EventMesh.publish("set_led_state", {"ai_state": "idle"}) + if TTS_str == None: + self.websocket_error_count+=1 + # if self.websocket_error_count <=3: + # EventMesh.publish("audio_montage_play",["xinhaoruo1.mp3"]) + if play_weak_signal_tone and self.websocket_error_count < 10: + EventMesh.publish("audio_montage_play", ["xinhaoruo1.mp3"]) + elif self.websocket_error_count >=10: + # EventMesh.publish("paly_iccid_imei") + EventMesh.publish("audio_montage_play", ["xinhaoruo1.mp3"]) + # else: + # EventMesh.publish("audio_montage_play",["R_SIM_FAILE.mp3"]) + #self.aud.play(1,1,'U:/static/R_SIM_FAILE.mp3') + if self.websocket_error_count <3 : + utime.sleep_ms(3000) + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + else: + print("websocket_error:tts:{}".format(TTS_str)) + self.AI_TTS_str = TTS_str + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + self.Close_flag = 1 + print("websocket_error2{}".format(self.websocket_error_count)) + + + print("websocket_error4") + def websocket_deconnect_VC(self): + #print("websocket_deconnect_VC {}!!!!!!!".format(self.AI_client_VC_isconnect)) + print("websocket_deconnect_VC!!!!!!!") + self._vc_session_started = 0 + # 断链/模式切换时必须重置“上行录音状态”，否则会被 _uplink_request_start 的幂等门控挡住， + # 表现为：切回长对话后无法自动开麦，按键 cancel/canceled 也无法恢复。 + try: + self.VC_record_state_flag = 0 + except Exception: + pass + try: + self._server_vad_silence = 0 + except Exception: + pass + try: + self._ptt_audio_created = 0 + except Exception: + pass + try: + # 若录音线程/录音设备仍在占用，先停掉 + self.VC_Speech_End() + except Exception: + try: + self.VC_record_thread_close() + except Exception: + pass + # 断链必须同步重置 FSM，否则 await_played/turn 残留会导致动态切换后“无法自动开麦/按键无效” + try: + self.fsm.session = VCSessionState.Disconnected + self.fsm.turn = VCTurnState.Idle + self.fsm.await_played = False + self.fsm.last_tts_completed_ms = 0 + self.fsm.last_llm_ms = 0 + self.fsm.reset_rx_phases() + except Exception: + pass + self.AI_client_VC_recv_thread_close()#关闭读线程 + self.AI_client_VC_recv_id = 0 + if self.AI_client_VC_isconnect == 1 or self.AI_client_VC!=None: + try: + self.AI_client_VC_isconnect = 0 + count = 500 + while True:#等待发完数据 + count-=1 + if count <=0 or self.AI_VC_send_data_queue_thread_lock == 0: + break + utime.sleep_ms(1) + + try: + self.AI_client_VC.close() + self.AI_client_VC_debug -=1 + #self.AI_client_VC=None + except Exception as e: + usys.print_exception(e) + try : + self.AI_client_VC.close() + self.AI_client_VC_debug -=1 + #self.AI_client_VC=None + except Exception as e: + usys.print_exception(e) + try : + self.AI_client_VC._close() + except Exception as e: + usys.print_exception(e) + self.AI_client_VC = None + except Exception as e: + usys.print_exception(e) + print("websocket_deconnect_VC {}!!!!!!!".format(self.AI_client_VC_isconnect)) + def AI_client_VC_ping(self): + pass + if self.AI_client_VC_isconnect == 1: + #self.AI_client_VC.write_frame(0x09,b'ping ping') + self.AI_client_VC_send("ping") + if VC_HEARTBEAT_LOG: + print("VC: ping ping") + def AI_client_VC_send(self,data_str): + # if isinstance(data_str, str):#是字符串 + # print("AI_client_VC_send:{}".format(data_str)) + # pass + # else: + # print("AI_client_VC_send:len{}".format(len(data_str))) + # 统计“语音上行”真实开始/结束：这里只统计二进制音频（非 ping/字符串） + try: + if not isinstance(data_str, str): + l = len(data_str) + if l > 0: + self._uplink_mark_start(l) + self._uplink_bytes += l + except Exception: + pass + self.AI_VC_send_data_queue.put([AI_client_VC_num,data_str]) + def AI_VC_send_data_queue_thread_close(self): + if self.AI_VC_send_data_queue_thread_id != 0: + # 先清空待发队列，避免关闭过程持续堆积/刷屏 + if self.AI_VC_send_data_queue.empty() != True: + self.AI_VC_send_data_queue.clear() + # 通知发送线程退出；若发送线程正阻塞在底层 send，则只能等超时或强制 stop_thread + self.AI_VC_send_data_queue_thread_lock = 2 + count = 500 + while True: + count -= 1 + if self.AI_VC_send_data_queue_thread_lock == 0 or count <=0: + break + else: + # 该打印只是“等待线程停下来”的调试心跳，默认关闭以免影响系统实时性 + if VC_HEARTBEAT_LOG and (count % 50 == 0 or count > 490): + print("AI_VC_send_data_queue_thread going:{}...".format(count)) + utime.sleep_ms(5) + self.AI_VC_send_data_queue_thread_lock = 0 + if VC_HEARTBEAT_LOG: + print("AI_VC_send_data_queue_thread_close") + if self.AI_VC_send_data_queue_thread_id != 0: + try: + # 线程可能已自然退出/被复用，stop_thread 会抛 ValueError + try: + running = _thread.threadIsRunning(self.AI_VC_send_data_queue_thread_id) + except Exception: + running = True + if running: + _thread.stop_thread(self.AI_VC_send_data_queue_thread_id) + except Exception as e: + usys.print_exception(e) + self.AI_VC_send_data_queue_thread_id = 0 + def AI_VC_send_data_queue_thread(self): + print("AI_VC_send_data_queue_thread:start") + count_t = 0 + while True: + if self.AI_VC_send_data_queue_thread_lock == 2: + print("AI_VC_send_data_queue_thread,该退出了") + break + if self.AI_VC_send_data_queue.empty() :#不能去掉，去掉会卡死 + self.AI_VC_send_data_queue_thread_lock = 0 + count_t+=1 + # if count_t%200 == 0: + # print("self.AI_VC_send_data_queue.empty") + utime.sleep_ms(1) + else: + #data[0]是websocket连接对象编号，data[1]是数据 + #print("AI_VC_send_data_queue_thread:get") + + self.AI_VC_send_data_queue_thread_lock = 1 + data = self.AI_VC_send_data_queue.get() + + + #print("AI_VC_send_data_queue_thread:num{} len{}".format(data[0],len(data[1]))) + if data != None: + try: + #data["func"](data["data"]) + # 发送数据 + #bytes_data = bytes(data[1]) + #data[0].send(data[1]) + #print("AI_VC_send_data_queue_thread:len:{}".format(len(data[1]))) + #print("AI_VC_send_data_queue_thread on line asr recv :{} vc recv:{}".format(_thread.threadIsRunning(self.AI_client_asr_recv_id),_thread.threadIsRunning(self.AI_client_VC_recv_id))) + if data[0] == AI_client_VC_num: + + if isinstance(data[1], str):#是字符串 + self.VC_pingping_timerout = 0 + s = data[1] + if s is None: + s = "" + # 保活 ping 不落 WS IO 日志（避免周期性刷屏） + if VC_WS_IO_LOG and s != "ping": + try: + if len(s) > 240: + preview = s[:160] + " ... " + s[-60:] + else: + preview = s + print("[VC] WS send_txt len={} txt={}".format(len(s), preview)) + except Exception: + try: + print("[VC] WS send_txt {}".format(data[1])) + except Exception: + pass + elif not VC_WS_IO_LOG: + if s == "ping": + pass + else: + try: + ev, tid = _vc_out_json_event_meta(s) + if ev: + print("[VC][proto] tx event={} trace_id={} len={} ws_conn={}".format( + ev, tid or "-", len(s), self.AI_client_VC_isconnect)) + else: + print("[VC][proto] tx unknown_json len={} ws_conn={}".format( + len(s), self.AI_client_VC_isconnect)) + except Exception: + print("[VC][proto] tx parse_fail len={}".format(len(s))) + else: + pass + # if VC_AUDIO_FRAME_LOG: + # print("VC: send_data len:{}".format(len(data[1]))) + # print("self.AI_client_VC = {} , self.AI_client_VC_isconnect = {}".format(self.AI_client_VC,self.AI_client_VC_isconnect)) + if self.AI_client_VC != None and self.AI_client_VC_isconnect == 1: + self.AI_client_VC.send(data[1]) + else: + try: + if isinstance(data[1], str) and data[1] != "ping": + _ev, _tid = _vc_out_json_event_meta(data[1]) + print("VC VC: send_data:ERROR len={} event={} trace_id={}".format( + len(data[1]), _ev or "?", _tid or "-")) + else: + print("VC VC: send_data:ERROR:{}".format(len(data[1]))) + except Exception: + print("VC VC: send_data:ERROR:{}".format(len(data[1]))) + #print("VC: send_data end") + + #print("VC: send_data end") + except Exception as e: + print("DeviceInfo_queue_thread:ERROR:{}".format(e)) + usys.print_exception(e) + if data[0] == AI_client_VC_num: + #self.websocket_error() + pass + + + try: + if self.AI_client_VC_isconnect != 0: + print("VC发送数据错误，退出") + EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_error,"data":"VC发送出错了，请重试"}) + self.ud_error(["VC",self.VC_taskid]) + except Exception as e: #网络连接异常 + usys.print_exception(e) + + utime.sleep_ms(1) + self.AI_VC_send_data_queue_thread_lock = 0 + def AI_client_VC_recv_thread_close(self): + if self.AI_client_VC_recv_id != 0: + count =500 + while True: + count -= 1 + if self.AI_client_VC_recv_lock == 1: + self.AI_client_VC_recv_lock = 2 + if self.AI_client_VC_recv_lock == 0 or count <=0: + break + else: + print("AI_client_VC_recv_thread going:{}...".format(count)) + utime.sleep_ms(1) + self.AI_client_VC_recv_lock = 0 + print("AI_client_VC_recv_thread_close") + if self.AI_client_VC_recv_id!= 0: + try: + _thread.stop_thread(self.AI_client_VC_recv_id) + except Exception as e: + usys.print_exception(e) + self.AI_client_VC_recv_id = 0 + def AI_client_VC_recv(self,cli): + count = 0 + asr_led_flag = 0 + tts_led_flag = 0 + while True: + try: + # 死循环接收数据 + self.AI_client_VC_recv_lock = 0 + + count+=1 + if self._vc_tts_pipeline_busy() and (self.tts_in_data_len - self.tts_out_data_len > 64*1024 or count>5): + count = 0 + utime.sleep_ms(1) + if not self._vc_tts_pipeline_busy(): + utime.sleep_ms(1) + recv_data = cli.recv() + if self.AI_client_VC_recv_lock == 2: + print("AI_client_VC_recv 该退出了") + break + self.AI_client_VC_recv_lock = 1 + if self._vc_rx_resets_long_idle_timer(recv_data): + self.close_timerout = 0 + if recv_data == "websocket close": + # 服务器关闭连接 + print("VC 服务器关闭连接!!!!!!!!!!!!!!!!!!!!!!!!!!!") + #self.AI_client_VC_isconnect = 0 + #self.websocket_deconnect_VC() + #EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_asr}) + # #EventMesh.publish("publish_AI_btnInfo",{"func":utime.sleep_ms,'data':(1500)}) + #EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_connect_asr}) + EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_error,"data":"VC连接断开了，请重试"}) + self.ud_error(["VC",self.VC_taskid]) + break + elif recv_data == "":#客户端关闭连接 + print("VC VC :客户端关闭连接") + EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_error,"data":"VC连接断开了，请重试"}) + self.ud_error(["VC",self.VC_taskid]) + break + try: + self.websocket_error_count = 0 + if isinstance(recv_data, str):#返回的是字符串 + jsload_data = ujson.loads(recv_data) + if VC_WS_IO_LOG: + print("[VC] recv raw:{}".format(recv_data)) + else: + try: + _rev = jsload_data.get("event", "") + if _rev != "llm.delta": + print("[VC] rx event={} sid={}".format( + _rev, jsload_data.get("sid", "") or "-")) + except Exception: + pass + #print("AI_client_VC_recv jsload_data = {}".format(jsload_data)) + try: + self._cur_sid = jsload_data.get("sid", "") or "" + except Exception: + self._cur_sid = "" + try: + ev = jsload_data.get("event", "") + if ev: + self._vc_dispatch_server_event(ev, jsload_data) + except Exception: + pass + + if jsload_data["event"] == "started" :#会话开始 + #payload = jsload_data["payload"] + # self.VC_dialog_id = payload["dialog_id"] + # self.VC_task_id = jsload_data["task_id"] + # started 仅表示链路就绪：不在这里提前切灯态； + # 聆听灯只在“录音真正开始/停止”处更新（VC_Speech_Start/End）。 + # 会话开始：允许上行；连续对话模式下直接进入“可发送音频”状态 + self._vc_session_started = 1 + # started 时不要强制清 Close_flag： + # - Close_flag==1：提示音/模式切换播完退出 + # - Close_flag==0：正常对话流程 + self._ptt_audio_created = 0 + self._server_vad_silence = 0 + self._turn_has_asr_completed = 0 + self._turn_has_vad_silence = 0 + self._turn_has_tts_started = 0 + self._llm_completed_ms = 0 + if self.VC_mode == 0 and getattr(self, "Close_flag", 0) == 0: + _skip_uplink = 0 + try: + if getattr(self, "_skip_started_uplink_once", 0) == 1: + self._skip_started_uplink_once = 0 + _skip_uplink = 1 + print("[VC][uplink_ctrl] Deny START by server.started cause=boot_prompt") + except Exception: + pass + if not _skip_uplink: + # 会话已 started：若此时上行未启动，则在这里补一次开麦，避免“Start 前开麦被 Close 停掉”后 + # 出现首轮无音频上行导致“第一次对话没有”的情况。 + if self.record_data_save_flag == 0: + if self.VC_record_state_flag == 0: + self.request_uplink_start("server.started") + else: + self._vc_clear_stale_record_state("server.started") + self.request_uplink_start("server.started/retry") + + print("VC:Started!!!!!!!!\r\n") + print("TTS音频为：{}，语音下发地址为：{}".format(self.AI_TTS_str,self.AI_new_voice_url)) + if self.AI_new_voice_url != "" and self.alarm_play_flag == 1 :#有闹钟,先响闹钟 + self.AI_VC_start() + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放铃声 + self.alarm_play_flag = 0 + self.voice_choose_flag = 1 + elif self.AI_TTS_str == "" and self.AI_new_voice_url != "" and self.music_play_flag == 0: + # if EventMesh.publish("get_history_voice_flag") == 1: + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 + # self.recv_voice = 0#当留言列表为空，留言播放标志清空 + # else: + # self.voice_cnt = 0 + # print("定时器开启222222222") + # self.voice_timer.start(1000,1,self.voice_judge) + elif self.AI_TTS_str != "" and self.AI_new_voice_url == "": + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_synthesis}) + + elif self.AI_TTS_str != "" and self.AI_new_voice_url != "" and self.music_play_flag == 0:#遇到tts和语音留言冲突时(并且没有音乐播放) + if EventMesh.publish("get_history_voice_flag") == 1:#播放历史留言 + if EventMesh.publish("get_wait_voice_flag") == 0: + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_synthesis}) + self.his_cnt = 0 + self.his_timer.start(1000,1,self.history_play) + self.voice_choose_flag = 1 + else:#等待留言 + if self.pwk_notice_flag == 0: + self.wait_cnt = 0 + print("未听播放1") + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_synthesis}) + self.wait_timer.start(1000,1,self.wait_play) + self.voice_choose_flag = 1 + + else:#已确定播放 + print("未听播放2") + self.wait_cnt = 0 + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_synthesis}) + self.wait_timer.start(1000,1,self.wait_single_play) + self.voice_choose_flag = 1 + else:#播放正常留言 + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_synthesis}) + self.voice_cnt = 0 + print("定时器开启111111111") + self.voice_timer.start(1000,1,self.voice_judge) + self.voice_choose_flag = 1 + # self.AI_TTS_str = "" + elif self.AI_TTS_str != "" and self.AI_new_voice_url != "" and self.music_play_flag == 1 :#遇到tts和语音留言冲突时(并且有音乐播放) + self.music_play_flag = 0 + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_synthesis}) + self.music_timer.start(1000,1,self.music_play) + self.voice_choose_flag = 1 + else: + #开始上传语音 + try: + if "vc_config" in globals() and vc_config and vc_config.get_bool("UPLOAD_START_TONE_ENABLED", False): + self.chenggong_tone() + except Exception: + pass + if self.VC_mode == 0: + self.request_uplink_start("server.started/idle") + else: + # Push-To-Talk：必须先同步发出 audio.create，再开麦。 + # request_send_audio_create() èµ° EventMesh 异步队列，会导致录音线程先于 + # audio.create 上行，深度睡眠唤醒后首轮 ASR 空结果、第二轮才正常。 + try: + self.AI_client_VC_send_speech() + except Exception: + self.request_send_audio_create() + # 若长按发生在“未 started”窗口，收到 started 后立即开麦进入新一轮 PTT + try: + if getattr(self, "_ptt_force_record_on_started", 0) == 1: + self._ptt_force_record_on_started = 0 + self.VC_Speech_Start_now() + except Exception: + pass + + + # if self.AI_new_voice_url != "": + # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice}) + + elif jsload_data["event"] == "config.update" :#配置更新中，需要停止录音 + try: + self.fsm.reset_rx_phases() + except Exception: + pass + self.request_uplink_stop("config.update") + + elif jsload_data["event"] == "asr.started" :#asr开始 + try: + self.fsm.on_server_asr_started() + except Exception: + pass + self._server_vad_silence = 0 + self._turn_has_asr_completed = 0 + self._turn_has_vad_silence = 0 + self._turn_has_tts_started = 0 + self._llm_completed_ms = 0 + try: + self._asr_completed_tone_done = 0 + except Exception: + pass + # 单对话：_drop_tts_audio 在 VC_Speech_Start 真开录后再清 0，避免 canceled 后仍播旧 TTS + try: + if getattr(self, "VC_mode", 0) != 1: + self._drop_tts_audio = 0 + except Exception: + pass + # 单对话：勿在 asr.started ç½® flag=2（未真开麦会僵死）；由 VC_Speech_Start 维护 + self.LocalRespondingEnded_timerout = 0 + elif jsload_data["event"] == "asr.result" :#asr结果 + try: + self.AI_ASR_str = jsload_data["payload"]["content"] + except Exception: + try: + self.AI_ASR_str = jsload_data.get("payload", {}).get("text", "") or "" + except Exception: + self.AI_ASR_str = "" + print("[VC][ASR] {}".format(self.AI_ASR_str)) + try: + if (self.AI_ASR_str or "").strip(): + self._last_asr_result_text = self.AI_ASR_str + except Exception: + pass + if self.AI_ASR_str == "":#没识别到 + self.AI_TTS_str = "我没听清，请再说一遍" + self._vc_led_alert_rb_flash() + elif jsload_data["event"] == "asr.completed" :#asr 终止的标识 + self._turn_has_asr_completed = 1 + try: + if not (self.AI_ASR_str or "").strip(): + self.AI_ASR_str = getattr(self, "_last_asr_result_text", "") or "" + except Exception: + pass + # 本地 think 提示：仅在有有效 ASR 文本时播，避免空轮次连播「滴」 + try: + if getattr(self, "_asr_completed_tone_done", 0) == 1: + pass + elif not self._vc_effective_asr_text(): + try: + print("[VC][ASR] completed empty, skip think tone") + except Exception: + pass + else: + stream_on = False + try: + stream_on = EventMesh.publish("get_vc_tts_playback_active") == 1 + except Exception: + stream_on = False + pcm_busy = getattr(self, "play_pcm", None) is not None + playing = getattr(self, "VC_Play_flag", 0) == 1 + try: + thinking = getattr(self.fsm, "turn", VCTurnState.Idle) == VCTurnState.Thinking + except Exception: + thinking = False + if (not stream_on) and (not pcm_busy) and (not playing) and (not thinking): + self._vc_local_play('U:/static/think.mp3', 1, 1, 650) + self._asr_completed_tone_done = 1 + except Exception: + pass + print("self.commands_mode:{}".format(self.commands_mode)) + if self.commands_mode ==2: + ret= self.check_text(self._vc_effective_asr_text()) + if ret == 0:#配置指令 + + if self.AI_VC_send_data_queue.empty()!=True: + self.AI_VC_send_data_queue.clear() + if self.AI_ASR_send_data_queue.empty()!=True: + self.AI_ASR_send_data_queue.clear() + self.AI_ASR_send_buf_len = 0 + EventMesh.publish("set_led_state", {"ai_state": "thinking"}) + try: + + self.request_uplink_stop("asr.completed/config_cmd") + EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + except Exception as e: + usys.print_exception(e) + continue + elif ret == 1:#不处理 + pass + #self.VC_Speech_End() + #EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) + + # if self.record_data_save_flag == 1: + # if self.record_fd : + # self.record_fd.close() + # self.record_fd = None + + # if dev_type != "7PIN" : + #EventMesh.publish("publish_AI_btnInfo",{"func":self.send_tone}) + elif jsload_data["event"] == "vad.active" :#VAD 检测到说话→聆听 + try: + if "vc_config" in globals() and vc_config and vc_config.get_bool("VAD_ACTIVE_BLUE_LED", True): + self._server_vad_silence = 0 + except Exception: + pass + elif jsload_data["event"] == "vad.silence" :#VAD 终止的标识 + # 说明服务端已判定“本轮说话结束”；此时继续上行只会干扰后续（例如推迟 TTS）。 + # 仅在“正在录音上行”的语音阶段才停止，避免误触发。 + self._server_vad_silence = 1 + self._turn_has_vad_silence = 1 + self._uplink_mark_end("vad.silence") + if self.record_data_save_flag == 0: + if self.VC_record_state_flag != 0 and self._vc_vad_uplink_stop_eligible(): + self.request_uplink_stop("vad.silence") + elif jsload_data["event"] == "llm.created" :#思考（LLM） + try: + self._ptt_await_server = 0 + except Exception: + pass + try: + EventMesh.publish("set_led_state", {"ai_state": "thinking"}) + except Exception: + pass + self._llm_completed_ms = 0 + # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Play_Start}) + pass + elif jsload_data["event"] == "llm.completed" :#llm完成（可能无 tts） + try: + self._llm_completed_ms = utime.ticks_ms() + except Exception: + # 勿用 1：ticks_diff(now,1)≈开机至今，会瞬间满足 thinking 超时误以为该重启 + self._llm_completed_ms = 0 + try: + self._cur_dialog_id = jsload_data.get("payload", {}).get("dialog_id", "") or "" + except Exception: + self._cur_dialog_id = "" + try: + self.fsm.log("server.llm.completed") + except Exception: + pass + pass + elif jsload_data["event"] == "llm.delta" :#llm返回数据 + if self.is_VAD == 0 and self.record_data_save_flag == 0: + if self.VC_record_state_flag == 1:#llm返回数据还没停止，就先停止 + self.request_uplink_stop("llm.delta") + elif jsload_data["event"] == "tts.started" :#tts开始 + try: + self._ptt_await_server = 0 + except Exception: + pass + self._server_vad_silence = 1 + self._turn_has_tts_started = 1 + self._tts_pending = 0 + self._tts_pending_ms = 0 + self._uplink_mark_end("tts.started") + # stop uplink + play_start 由 FSM action list 统一触发，避免多处重复调用 + # 标记本轮 TTS 来源： + # - 有 payload.dialog_id：认为是“对话回复 TTS” + # - 无 dialog_id：认为是“提示/提醒 TTS”（模式切换可通过 _pending_tts_kind 标记） + try: + did = jsload_data.get("payload", {}).get("dialog_id", "") or "" + except Exception: + did = "" + # 注意：无 dialog_id 的 tts.started 很可能是提示音/提醒音； + # 但“模式切换提示音”的 kind 可能已在 synthesis 时写入 _last_tts_kind， + # 这里不应无条件覆盖为 prompt，否则播完无法触发下一轮开麦。 + try: + if did: + self._last_tts_kind = "dialog" + else: + k = getattr(self, "_pending_tts_kind", "") or "" + if k: + self._last_tts_kind = k + else: + cur = getattr(self, "_last_tts_kind", "") or "" + if cur in ( + "mode_switch_long", + "mode_switch_ptt", + "boot_prompt", + ): + pass + elif not cur: + self._last_tts_kind = "prompt" + except Exception: + pass + try: + self._pending_tts_kind = "" + except Exception: + pass + # 播放灯由阶段事件控制 + try: + self._tts_completed_wall_ms = 0 + except Exception: + pass + elif jsload_data["event"] == "tts.completed" :#tts结束 + # tts.completed 仅表示下发结束，本地可能仍在播放；绿灯关闭在本地播放结束时处理 + try: + # server event 已在统一入口 on_server_event 触发 FSM，这里避免重复驱动 + pass + except Exception: + pass + try: + try: + sid = getattr(self, "_cur_sid", "") + did = getattr(self, "_cur_dialog_id", "") + except Exception: + sid, did = "", "" + print("[TTS] completed in_len={} out_len={} q_empty={} play_pcm={} sid={} did={}".format( + getattr(self, "tts_in_data_len", 0), + getattr(self, "tts_out_data_len", 0), + 1 if self.AI_audio_queue.empty() else 0, + 1 if self.play_pcm != None else 0, + sid, + did, + )) + except Exception: + pass + try: + self._tts_completed_wall_ms = utime.ticks_ms() + except Exception: + self._tts_completed_wall_ms = 0 + # 播完上报完全由 FSM await_played + play_end 驱动，这里不再使用 LocalRespondingEnded_timerout/_need_audio_played_ack + pass + + elif jsload_data["event"] == "canceled" :#打断 + # 停播/清队列已在 _vc_dispatch_server_event(canceled) 完成；此处仅收尾标志 + self.is_ruppt_tts = 0#设备上发中断tts，收到服务器返回的cancel后置为0 + self._need_audio_played_ack = 0 + self._tts_pending = 0 + self._tts_pending_ms = 0 + # 开麦由 FSM action list 统一触发（uplink_start:canceled），避免重复调用 + if self.AI_audio_queue_thread_id ==0: + _thread.stack_size(24*1024) + self.AI_audio_queue_thread_id = _thread.start_new_thread(self.AI_audio_queue_thread, ()) + _thread.stack_size(8*1024) + print("AI_VC_start :AI_audio_queue_thread_id :{} ".format(self.AI_audio_queue_thread_id)) + else: + print("AI_VC_start :AI_audio_queue_thread_id != 0 ") + elif jsload_data["event"] == "completed" :#会话结束（可能无 TTS，如 ASR 空） + # 重要：服务端常在 tts.completed 后紧跟 completed。 + # 若本轮存在 TTS/待发 audio.played，则不能在此清零 LocalRespondingEnded_timerout/_need_audio_played_ack， + # 否则会掐断「播完→audio.played→下一轮录音」链路，表现为最多 1~2 轮就停。 + if self._need_audio_played_ack == 1 or self._vc_tts_pipeline_busy() or self.VC_Play_flag == 1: + pass + else: + # 无 TTS 的 completed：清状态并按需进入下一轮（见下方条件） + self.LocalRespondingEnded_timerout = 0 + self._need_audio_played_ack = 0 + try: + self.fsm.reset_rx_phases() + except Exception: + pass + # 连续对话：若本轮没有 TTS（或服务端直接 completed），需要立刻回到聆听状态 + # 否则会出现“有时 1 è½®/2 轮后就停住”的现象（服务端 completed 但端侧未重新开录音上行）。 + if self.VC_mode == 0 and self.Close_flag == 0: + # 仅当“本轮已结束一次语句（vad.silence 或 asr.completed）且从未进入 tts.started” + # 才自动进入下一轮；避免把真正的“会话结束/异常结束”误当成下一轮入口。 + if (self._turn_has_tts_started == 0) and (self._turn_has_vad_silence == 1 or self._turn_has_asr_completed == 1): + self._server_vad_silence = 0 + # 开麦由 FSM action list 统一触发（uplink_start:completed），这里不再重复调用 + pass + + elif jsload_data["event"] == "command" :#命令匹配 + ret = -1 + try: + if self.commands_mode ==1: + if len(jsload_data["payload"]["functions"])>0: + + cmd = jsload_data["payload"]["functions"][0]["function"]["arguments"] + print("llm:cmd:{}".format(cmd)) + cmd = ujson.loads(cmd) + ret = self.check_text_llm(cmd) + elif self.commands_mode ==0: + + cmd = jsload_data["payload"] + print("server:cmd:{}".format(cmd)) + ret = self.check_text_server(cmd) + else: + cmd = jsload_data["payload"]["functions"][0]["function"]["arguments"] + print("llm:cmd:{}".format(cmd)) + cmd = ujson.loads(cmd) + ret = self.check_text_llm(cmd) + except Exception as e: + usys.print_exception(e) + if ret == -1: + if self.AI_TTS_str =="": + self.AI_TTS_str = "抱歉，没找到对应操作，可以换个说法试试" + #EventMesh.publish("publish_AI_btnInfo",{"func":utime.sleep_ms,'data':(10)}) + self.request_uplink_stop("command") + EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + elif jsload_data["event"] == "failed":#接口返回错误 + if jsload_data["payload"]["error"] == "concurrency_limit_exceeded": + EventMesh.publish("publish_AI_btnInfo",{"func":self.concurrency_limit_exceeded_tone}) + + #utime.sleep_ms(4000) + + else: + EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_error,"data":jsload_data["payload"]["message"]}) + pass + + elif jsload_data["event"] == "audio.convert.completed":#音频转成完成没有出错 接口 + # self.VC_state = VC_State_End + print("未读留言初始数值：{}".format(self.pwk_cnt_voice)) + if self.pwk_notice_flag == 1:#只有在播放的时候才会去计数 + if self.pwk_cnt_voice > 0: + self.pwk_cnt_voice -= 1#减1 + print("未读留言又-1,当前留言剩余数量为:{}".format(self.pwk_cnt_voice)) + else: + print("未读列表为空") + if self.pwk_cnt_voice == 0:#积累的留言已经全部播完 + self.pwk_notice_flag = 0 + self.wait_sort = 0 + self.wait_first_flag = 0#让下次未读播放提示恢复 + self.voicerupt_flag = 0 #标志位置0 + self.wait_voicerupt_flag = 0 + self.voice_notice = 1#开启回复 + if self.history_voice_flag == 1 and self.wait_voice_flag == 0: + self.voice_notice = 0 + EventMesh.publish("set_storage_config",["is_noread_voice",0]) + + if self.voice_choose_flag == 1:#播放完语音后，有冲突的时候关闭 + self.voice_choose_flag = 0 + self.Close_flag = 1 + self.LocalRespondingEnded_timerout = 8 + # EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) + if self.voice_notice == 1:#已收到留言，进行该留言回复逻辑处理 + tts_play_str = "如需回复，长按确认键开始录制，松开发送。十秒内无操作自动退出" + EventMesh.publish("play_tts",[tts_play_str,1]) + self.voice_notice = 0#标志位清空 + self.voice_timer.stop() + self.revoice_cnt = 0 + self.voice_flag = 1 + print("语音留言回复停止") + self.revoice_timer.start(1000,1,self.revoice_judge) + print("是否未读：{},是否播完：{}".format(EventMesh.publish("get_wait_voice_flag"),len(EventMesh.publish("get_play_new_voice_list")))) + # if EventMesh.publish("get_wait_voice_flag") == 1 and len(EventMesh.publish("get_play_new_voice_list")) ==0:#还在未读并且列表为空 + # self.wait_first_flag = 0 + # print("未读留言播完，清空标志") + # EventMesh.publish("set_wait_voice_flag",0) + # EventMesh.publish("set_history_voice_flag",0) + EventMesh.publish("update_play_new_voice_info")#更新已读状态 + self.recv_voice = 0 + print("不开启定时器标志位置0") + self.LocalRespondingEnded_timerout = 8 + try: + self.fsm.reset_rx_phases() + except Exception: + pass + + pass + elif jsload_data["event"] == "audio.convert.failed": #音频转换失败了 + pass + + + else:#下发音频数据 + #self.AI_ASR_str = ""#下发音频后清掉asr结果 + + + # PTT 打断后丢弃残留 TTS 音频帧，避免“断断续续继续播” + try: + if self._ptt_should_drop_inbound_tts(): + continue + except Exception: + pass + if self.VC_stop_play_audio_flag == 0: + print("新的AI对话不允许以前的播放数据:") + break + + # if VC_AUDIO_FRAME_LOG: + # print("下发音频数据1:{}".format(len(recv_data))) + + if self.play_data_save_flag == 1: + if self.play_fd ==None: + try: + uos.remove('/usr/play_data.pcm') + except Exception as e: + usys.print_exception(e) + self.play_fd = open('/usr/play_data.pcm','w') + if self.play_fd !=None: + self.play_fd.write(recv_data) + # print("下发音频数据2:{}".format(len(recv_data))) + #if self.VC_state >= VC_State_TTSStart: + #self.put_record_data([recv_data,""]) + if self.is_ruppt_tts == 0:#被设备主动打断打断则不播放，知道收到服务器返回的cancel才修改标志位 + if self.VC_play_TYPE == VC_play_TYPE_PCM: + if self.play_pcm != None: + try: + #print("play_pcm.stream_write !!!") + buf_len = self.play_pcm.stream_get_buf_len() + count_tmp = 1000 + while buf_len < len(recv_data): + utime.sleep_ms(1) + count_tmp-=1 + if count_tmp<=0 or self.play_pcm == None: + break + buf_len = self.play_pcm.stream_get_buf_len() + #print("buf已满，等待释放：buf_len:{},count_tmp:{}".format(buf_len,count_tmp)) + except Exception as e: + usys.print_exception(e) + EventMesh.publish("publish_AI_btnInfo",{"func":self.put_record_data,"data":[recv_data,""]}) + else: + print("被打断不播放") + utime.sleep_ms(1) + except Exception as e: + usys.print_exception(e) + + except Exception as e: + error_str = str(e) + #print("VC VC:e:{}".format(error_str)) + if error_str.find("[Errno 110] ETIMEDOUT")!= -1 or error_str.find("[Errno 115] ")!= -1: + if VC_HEARTBEAT_LOG: + print("VC VC:recv 正常报错：{}".format(error_str)) + # recv 超时仅表示本周期无帧，链路仍可能在等 LLM/TTS；清零避免误触发「接口无响应」 + self.close_timerout = 0 + pass + else: + usys.print_exception(e) + #self.websocket_error() + print("VC VC :读数据报错，退出") + EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_error,"data":"VC出错了，请重试"}) + self.ud_error(["VC",self.VC_taskid]) + break + self.AI_client_VC_isconnect = 0 + self.AI_client_VC_recv_id = 0 + self.AI_client_VC_recv_lock = 0 + print("[VC] recv_thread exit") + def get_recv_voice(self,event,msg): + return self.recv_voice + def set_recv_voice(self,event,msg): + return self.recv_voice + def check_text_server(self,cmd): + return -1 + def _apply_vc_text_intent(self, text, cmd, llm_style_tts): + """表驱动文本意图（vc_text_intent）。未命中返回 None；命中返回 0/1/-1。""" + if vc_text_intent is None: + return None + try: + try: + _btv = bool("vc_config" in globals() and vc_config and vc_config.get_bool("BT_VOICE_FEATURES_ENABLED", False)) + except Exception: + _btv = False + r = vc_text_intent.match_vc_command(text, {"bt_voice": _btv}) + except Exception as e: + usys.print_exception(e) + return None + if not r: + return None + return self._apply_vc_text_intent_matched(r, cmd, llm_style_tts) + def _apply_vc_text_intent_matched(self, r, cmd, llm_style_tts): + if r.get("needs_payload") and cmd is None: + return None + it = r["intent"] + if r.get("tts"): + self.AI_TTS_str = r["tts"] + if r.get("close_flag"): + self.Close_flag = 1 + if it == "goodbye": + return 0 + if it == "shutdown": + if llm_style_tts: + EventMesh.publish("publish_AI_btnInfo", {"func": self.Close}) + EventMesh.publish("Power_off") + return 1 + try: + close_only = ("vc_config" in globals() and vc_config and vc_config.get_bool("POWEROFF_CMD_CLOSE_ONLY", True)) + except Exception: + close_only = True + if close_only: + EventMesh.publish("publish_AI_btnInfo", {"func": self.Close}) + return 1 + EventMesh.publish("publish_AI_btnInfo", {"func": self.Close}) + EventMesh.publish("Power_off") + return 1 + if it == "toggle_dialog_mode": + if self.VC_mode == 0: + self.VC_mode = 1 + if llm_style_tts: + self.AI_TTS_str = "了解，已切换到单对话模式" + else: + try: + self.AI_TTS_str = vc_config.get_attr("CMD_TOGGLE_MODE_TO_PTT_TTS", "已切换到单对话模式") + except Exception: + self.AI_TTS_str = "已切换到单对话模式" + else: + self.VC_mode = 0 + if llm_style_tts: + self.AI_TTS_str = "了解，已切换到长对话模式" + else: + try: + self.AI_TTS_str = vc_config.get_attr("CMD_TOGGLE_MODE_TO_LONG_TTS", "已切换到长对话模式") + except Exception: + self.AI_TTS_str = "已切换到长对话模式" + EventMesh.publish("set_storage_config", ["VC_mode", self.VC_mode]) + return 0 + if it == "volume_set": + try: + pj = ujson.loads(cmd["command"]["params_json"]) + volume = pj["n"] + except Exception as e: + usys.print_exception(e) + try: + self.AI_TTS_str = cmd["command"]["fail_prompt"] + except Exception: + self.AI_TTS_str = "" + return -1 + if volume > 100 or volume < 0: + self.AI_TTS_str = "音量调整超出范围了，只能0-100哦" + else: + self.VC_TTS_volume = volume + self.aud.setVolume(self.VC_TTS_volume // 10) + EventMesh.publish("set_storage_config", ["VC_TTS_volume", self.VC_TTS_volume]) + self.AI_TTS_str = "了解，音量已经调整,当前音量是{}".format(self.VC_TTS_volume) + EventMesh.publish("post_device_state") + EventMesh.publish("adjust_volume_toBle", self.VC_TTS_volume) + return 0 + if it == "volume_up": + if llm_style_tts: + try: + print("self.VC_TTS_volume add:{}".format(self.VC_TTS_volume)) + except Exception: + pass + self.VC_TTS_volume += 10 + if self.VC_TTS_volume > 100: + self.AI_TTS_str = "音量已经是最大啦" if llm_style_tts else "音量已最大" + self.VC_TTS_volume = 100 + else: + self.aud.setVolume(self.VC_TTS_volume // 10) + EventMesh.publish("set_storage_config", ["VC_TTS_volume", self.VC_TTS_volume]) + EventMesh.publish("post_device_state") + EventMesh.publish("adjust_volume_toBle", self.VC_TTS_volume) + self.AI_TTS_str = ( + "了解，音量已经调大了,当前音量是{}".format(self.VC_TTS_volume) + if llm_style_tts + else "配置成功,当前音量 {}".format(self.VC_TTS_volume) + ) + return 0 + if it == "volume_down": + if llm_style_tts: + try: + print("self.VC_TTS_volume end:{}".format(self.VC_TTS_volume)) + except Exception: + pass + self.VC_TTS_volume -= 10 + if self.VC_TTS_volume < 0: + self.VC_TTS_volume = 0 + self.AI_TTS_str = "音量已经是最小啦" if llm_style_tts else "音量已最小" + else: + self.aud.setVolume(self.VC_TTS_volume // 10) + EventMesh.publish("set_storage_config", ["VC_TTS_volume", self.VC_TTS_volume]) + EventMesh.publish("post_device_state") + EventMesh.publish("adjust_volume_toBle", self.VC_TTS_volume) + self.AI_TTS_str = ( + "好嘞，音量已经调小了,当前音量是{}".format(self.VC_TTS_volume) + if llm_style_tts + else "配置成功,当前音量 {}".format(self.VC_TTS_volume) + ) + return 0 + if it in ("speech_faster", "speech_rate_up"): + self.speech_rate += 10 + if self.speech_rate > 50: + self.AI_TTS_str = "语速已经是最快了" + self.speech_rate = 50 + else: + EventMesh.publish("set_storage_config", ["speech_rate", self.speech_rate]) + self.AI_TTS_str = "明白，语速已经调快了,当前语速是{}".format(self.speech_rate) + return 0 + if it in ("speech_slower", "speech_rate_down"): + self.speech_rate -= 10 + if self.speech_rate < -50: + self.speech_rate = -50 + self.AI_TTS_str = "语速已经是最慢了" + else: + EventMesh.publish("set_storage_config", ["speech_rate", self.speech_rate]) + self.AI_TTS_str = "明白，语速已经调慢了,当前语速是{}".format(self.speech_rate) + return 0 + if it == "vad_faster": + self.vad_silence -= 200 + if self.vad_silence < 100: + self.AI_TTS_str = "已经是最快了" + self.vad_silence = 100 + else: + self.AI_TTS_str = "配置成功，当前语音断句速度是{}毫秒".format(self.vad_silence) + EventMesh.publish("set_storage_config", ["vad_silence", self.vad_silence]) + return 0 + if it == "vad_slower": + self.vad_silence += 200 + if self.vad_silence > 2000: + self.vad_silence = 2000 + self.AI_TTS_str = "已经是最慢了" + else: + self.AI_TTS_str = "配置成功，当前语音断句速度是{}毫秒".format(self.vad_silence) + EventMesh.publish("set_storage_config", ["vad_silence", self.vad_silence]) + return 0 + if it == "query_signal": + signal = net.csqQueryPoll() + self.AI_TTS_str = "当前信号{}".format(signal) + return 0 + if it == "query_battery": + battery = EventMesh.publish("get_battery") + try: + print("电量查询：{}".format(battery)) + except Exception: + pass + self.AI_TTS_str = "电量剩余百分之{}".format(battery) + return 0 + if it == "query_voltage": + Vbatt = Power.getVbatt() + self.AI_TTS_str = "当前电池电压{}".format(Vbatt) + return 0 + if it == "query_iccid": + iccid = sim.getIccid() + if iccid != -1 and len(iccid) > 5: + try: + print("iccid{}".format(iccid)) + except Exception: + pass + self.AI_TTS_str = "当前卡号为:{}".format(iccid) + else: + self.AI_TTS_str = "卡号查询失败" + return 0 + if it == "query_imei": + imei = modem.getDevImei() + if imei != -1 and len(imei) > 5: + try: + print("IMEI：{}".format(imei)) + except Exception: + pass + self.AI_TTS_str = "当前设备号为:{}".format(imei) + else: + self.AI_TTS_str = "设备号查询失败" + return 0 + if it == "query_version": + bt_versions = EventMesh.publish("get_bt_versions") + if bt_versions is None or bt_versions == "": + self.AI_TTS_str = "当前版本号:" + version_util.get_tts_version() + else: + self.AI_TTS_str = "当前版本号:" + version_util.get_tts_version(bt_versions) + return 0 + if it == "query_server": + self.AI_TTS_str = "服务器地址" + self.server_ip + return 0 + if it == "ota_check": + if self.ai_OTA.min_sys_ota() == 1: + self.AI_TTS_str = "有新版本，准备开始升级，升级过程会多次重启设备，请保证流量充足及电量充足" + else: + self.AI_TTS_str = "当前已是最新版本" + return 0 + if it == "factory_mode": + self.factory_mode() + return 1 + if it == "alarm_set": + try: + alarm = ujson.loads(cmd["command"]["params_json"]) + try: + print("params_json:{}".format(cmd["command"]["params_json"])) + except Exception: + pass + add_alarm_tmp = EventMesh.publish("AI_add_alarm", alarm["alarm"]) + if add_alarm_tmp[0] > 0: + self.AI_TTS_str = add_alarm_tmp[1] + return 1 + except Exception as e: + usys.print_exception(e) + try: + self.AI_TTS_str = cmd["command"]["fail_prompt"] + except Exception: + self.AI_TTS_str = "" + return -1 + if it == "alarm_clear": + try: + self.AI_TTS_str = EventMesh.publish("AI_delete_alarm") + except Exception as e: + usys.print_exception(e) + return 0 + if it == "alarm_list": + EventMesh.publish("read_alarm_list") + return 0 + if it == "voice_message_mode": + self.voice_flag = 1 + self.AI_TTS_str = "已进入语音留言模式，长按按钮开始录音，松开按钮结束" + self.Close_flag = 1 + self.stop_voice_timer.stop() + self.stop_voice_timer.start(30 * 1000, 0, self.exit_voice_message_timer) + try: + print("语音留言！！！！！！！！！！") + except Exception: + pass + return 0 + if it == "bt_ble_adv_on": + self.AI_TTS_str = "正在开蓝牙广播" + EventMesh.publish("AI_send_bt_ble_switch_toble", 1) + try: + print("正在开蓝牙广播") + except Exception: + pass + return 0 + if it == "bt_ble_adv_off": + self.AI_TTS_str = "正在关蓝牙广播" + EventMesh.publish("AI_send_bt_ble_switch_toble", 2) + try: + print("正在关蓝牙广播") + except Exception: + pass + return 0 + if it == "bt_classic_on": + self.AI_TTS_str = "正在开经典蓝牙" + EventMesh.publish("AI_send_bt_ble_switch_toble", 3) + try: + print("正在开经典蓝牙") + except Exception: + pass + return 0 + if it == "bt_classic_off": + self.AI_TTS_str = "正在关经典蓝牙" + EventMesh.publish("AI_send_bt_ble_switch_toble", 4) + try: + print("正在关经典蓝牙") + except Exception: + pass + return 0 + if it == "unbind_confirm": + self.AI_TTS_str = "设备解绑中...." + self.Close_flag = 1 + EventMesh.publish("send_unbind_dev_to_iot") + try: + print("解绑设备!!!!!!!!!!!!!!") + except Exception: + pass + return 0 + if it == "clear_earphone": + self.AI_TTS_str = "正在删除" + EventMesh.publish("clear_earphone_record") + try: + print("正在删除") + except Exception: + pass + return 0 + return None + def check_text_llm(self,cmd): + try: + desc = cmd["command"]["chinese_description"] + if desc.find("切换音色") != -1: + text = cmd["command"]["params_json"] + try: + for key in self.VC_TTS_voice_type: + if text.find(key) != -1: + self.VC_TTS_voice_key = key + EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) + self.AI_TTS_str = "了解，声音已切换" + return 0 + except Exception as e: + usys.print_exception(e) + if text.find("男") != -1: + if "男" in self.VC_TTS_voice_type: + self.VC_TTS_voice_key = "男" + EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) + self.AI_TTS_str = "了解，声音已切换" + return 0 + if text.find("女") != -1: + if "女" in self.VC_TTS_voice_type: + self.VC_TTS_voice_key = "女" + EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) + self.AI_TTS_str = "了解，声音已切换" + return 0 + if text.find("同声") != -1 or text.find("ç«¥") != -1: + if "ç«¥" in self.VC_TTS_voice_type: + self.VC_TTS_voice_key = "ç«¥" + EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) + self.AI_TTS_str = "了解，声音已切换" + return 0 + if text.find("粤") != -1 or text.find("越野") != -1 or text.find("广东话") != -1: + if "粤" in self.VC_TTS_voice_type: + self.VC_TTS_voice_key = "粤" + EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) + self.AI_TTS_str = "了解，声音已切换" + return 0 + if text.find("四川") != -1: + if "四川" in self.VC_TTS_voice_type: + self.VC_TTS_voice_key = "四川" + EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) + self.AI_TTS_str = "了解，声音已切换" + return 0 + if text.find("个性") != -1: + if "个性" in self.VC_TTS_voice_type: + self.VC_TTS_voice_key = "个性" + EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) + self.AI_TTS_str = "了解，声音已切换" + return 0 + if text.find("英语") != -1: + if "英语" in self.VC_TTS_voice_type: + self.VC_TTS_voice_key = "英语" + EventMesh.publish("set_storage_config", ["VC_TTS_voice_key", self.VC_TTS_voice_key]) + self.AI_TTS_str = "了解，声音已切换" + return 0 + self.AI_TTS_str = "很抱歉，没找到对应声音" + return 0 + ret = self._apply_vc_text_intent(desc, cmd, True) + if ret is not None: + return ret + except Exception as e: + usys.print_exception(e) + try: + self.AI_TTS_str = cmd["command"]["fail_prompt"] + except Exception: + self.AI_TTS_str = "" + return -1 + def check_text(self, text): + print("check_text:len:{}".format(len(text))) + if len(text) > 60: + return -1 + try: + if not isinstance(text, str): + return -1 + ret = self._apply_vc_text_intent(text, None, False) + if ret is not None: + return ret + except Exception as e: + usys.print_exception(e) + return -1 + def get_trace_id(self): + t = utime.mktime(utime.localtime()) + t = t*1000+urandom.randint(0, 1000) + md5_data = "{}".format(t)#md5签名 + sign_data = "" + hash_obj = uhashlib.md5(md5_data) + res = hash_obj.digest() + + # 转成十六进制表示 + sign_data = ubinascii.hexlify(res) + sign_data = sign_data.decode('ascii') + return sign_data + def get_session_id(self,data = "session_id"): + t = utime.mktime(utime.localtime()) + t = t*1000+urandom.randint(0, 1000) + md5_data = "{}{}".format(data,t)#md5签名 + sign_data = "" + hash_obj = uhashlib.md5(md5_data) + res = hash_obj.digest() + + # 转成十六进制表示 + sign_data = ubinascii.hexlify(res) + sign_data = sign_data.decode('ascii') + return sign_data + def get_new_message_id(self): + message_id = self.get_trace_id()+"aaa"+self.get_trace_id() + self.VC_message_id=message_id[:32] + return self.VC_message_id + def get_new_task_id(self): + task_id = self.get_trace_id()+"aaa"+self.get_trace_id() + self.VC_task_id=task_id[:32] + return self.VC_task_id + def AI_client_VC_start_requset(self): + voice="zh_male_wennuanahu_moon_bigtts" + try: + voice = self.VC_TTS_voice_type[self.VC_TTS_voice_key]["voice"] + except Exception as e: + usys.print_exception(e) + #voice="zh_female_dongmanxiaoyang_moon_bigtts" + token = EventMesh.publish("get_dev_token") + if token ==None: + token = "" + # bot_option：优先从 default_config（usr 可覆盖）读取 VC_BOT_ID / VC_WORKFLOW_ID，允许空字符串 + _def_bot_id = "7462617881076351002" + _def_workflow_id = "7462621757327196187" + try: + if "vc_config" in globals() and vc_config: + _vc_bot_id = vc_config.get_attr("VC_BOT_ID", _def_bot_id) + _vc_workflow_id = vc_config.get_attr("VC_WORKFLOW_ID", _def_workflow_id) + if _vc_bot_id is None: + _vc_bot_id = "" + if _vc_workflow_id is None: + _vc_workflow_id = "" + else: + _vc_bot_id, _vc_workflow_id = _def_bot_id, _def_workflow_id + except Exception: + _vc_bot_id, _vc_workflow_id = _def_bot_id, _def_workflow_id + json_data = \ + { + "header": { + "event": "Start", + "trace_id": self.get_trace_id(), + }, + "payload": { + "simple_response": True,#不返回llm过程文本 + "device_id": modem.getDevImei(), + + "user_agent": "AudioClient/1.0", + "dialog_attributes": { + #"provider": "doubao", + # "provider": "coze", + + "cfg": { + "disable_empty_prompt" : True,#拦截空字符请求大模型 + "custom_variables": { + "name": self.bot_name, + "devToken":token, + }, + "greeting":{ + "enabled":False, + "content":"" + }, + "openai_option":{ + }, + + "bot_option": { + "use_workflow": True, + #"input_format": "text", + "bot_id": _vc_bot_id, + "workflow_id": _vc_workflow_id, + + } + } + }, + "outbound_attributes": { + "provider": "volcengine", + "cfg": { + "format": "pcm", + "sample_rate": self.VC_play_samplerate, + "bit_rate": 16, + "channel": 1, + "volume": self.VC_TTS_volume, + "emotion": self.emotion, + #"voice": "7426720361732980745", + "voice": voice,#音色 + "speech_rate": self.speech_rate,#语速 + "chunk_interval_ms": 10, # 配置每段音频入队间隔时间 100ms + } + }, + "inbound_attributes": { + # "provider": "xun-fei",# + "provider": "volcengine", + "disable_vad":True, + "vad_silence":self.vad_silence, + "interrupt_options":{ + "concatenate_results":True, + "thinking_time_only": True, + #"keyword_interrupt": false, // 启用打断词功能 （打断词只有在思考以及之后才能打断） + #"interrupt_keywords": ["停止", "打断","打断一下", "等一下", "别说了", "打住"] // 配置打断词列表 + }, + "cfg": { + "format": "pcm", + "sample_rate": self.VC_record_samplerate, + "bit_rate": 16, + "channel": 1, + "enable_punctuation": True, + "enable_itn":True, + "keywords":[] + } + }, + "commands":self.commands, + } + } + # "bot_id":"7481164663863459850",#鸭子 + # "workflow_id":"7480485093845991443", + # "bot_id":"7462617881076351002",#语言助手 + # "workflow_id":"7462621757327196187", + # "bot_id":"7469964258592931852",#深圳百事通 + # "workflow_id":"7469968083063603241", + # "bot_id":"7411071327182438412",#柒牌 + # "workflow_id":"7469049680291610636", + # if self.online_kws == 1:#打开 + # json_data["payload"]["inbound_attributes"]["interrupt_options"]["keyword_interrupt"] = True + # json_data["payload"]["inbound_attributes"]["interrupt_options"]["interrupt_keywords"] = ["你好，柒先生","你好"] + #如果 + json_data["payload"]["outbound_attributes"]["provider"]=self.outbound_attributess_provider#TTS音色服务选择，火山 + json_data["payload"]["inbound_attributes"]["provider"]=self.inbound_attributess_provider#ASR服务选择，火山 + #优先选择self.VC_TTS_voice_type[self.VC_TTS_voice_key]里得参数 + # if "inbound_attributess_provider" in self.VC_TTS_voice_type[self.VC_TTS_voice_key]: + # if len(self.VC_TTS_voice_type[self.VC_TTS_voice_key]["inbound_attributess_provider"])>0: + # json_data["payload"]["inbound_attributes"]["provider"]=self.VC_TTS_voice_type[self.VC_TTS_voice_key]["inbound_attributess_provider"] + # print("inbound_attributess_provider是VC_TTS_voice_type里的：{}".format(self.VC_TTS_voice_type[self.VC_TTS_voice_key]["inbound_attributess_provider"])) + if "outbound_attributess_provider" in self.VC_TTS_voice_type[self.VC_TTS_voice_key]: + if len(self.VC_TTS_voice_type[self.VC_TTS_voice_key]["outbound_attributess_provider"])>0: + json_data["payload"]["outbound_attributes"]["provider"]=self.VC_TTS_voice_type[self.VC_TTS_voice_key]["outbound_attributess_provider"] + print("outbound_attributess_provider是VC_TTS_voice_type里的：{}".format(self.VC_TTS_voice_type[self.VC_TTS_voice_key]["outbound_attributess_provider"])) + + # 根据配置设置默认 bot/workflow，并允许 VC_TTS_voice_type 覆盖 + try: + if "vc_config" in globals() and vc_config: + opt = vc_config.get_attr("DIALOG_BOT_OPTION", None) + if isinstance(opt, dict): + if "bot_id" in opt: + json_data["payload"]["dialog_attributes"]["cfg"]["bot_option"]["bot_id"] = opt.get("bot_id") or "" + if "workflow_id" in opt: + json_data["payload"]["dialog_attributes"]["cfg"]["bot_option"]["workflow_id"] = opt.get("workflow_id") or "" + if "use_workflow" in opt: + json_data["payload"]["dialog_attributes"]["cfg"]["bot_option"]["use_workflow"] = bool(opt.get("use_workflow")) + except Exception: + pass + + try: + # voice 配置可覆盖 bot_id/workflow_id/openai_option + cur = self.VC_TTS_voice_type[self.VC_TTS_voice_key] + if isinstance(cur, dict): + if cur.get("bot_id"): + json_data["payload"]["dialog_attributes"]["cfg"]["bot_option"]["bot_id"] = cur.get("bot_id") + if "workflow_id" in cur: + json_data["payload"]["dialog_attributes"]["cfg"]["bot_option"]["workflow_id"] = cur.get("workflow_id") or "" + if cur.get("openai_option"): + json_data["payload"]["dialog_attributes"]["cfg"]["openai_option"] = cur.get("openai_option") + except Exception: + pass + + try: + _bo = json_data["payload"]["dialog_attributes"]["cfg"]["bot_option"] + if not _bo.get("bot_id", "") or not _bo.get("workflow_id", ""): + _bo["use_workflow"] = False + except Exception: + pass + try: + if "vc_config" in globals() and vc_config: + p = vc_config.get_attr("DIALOG_PROVIDER_DEFAULT", None) + if p: + json_data["payload"]["dialog_attributes"]["provider"] = p + except Exception: + pass + + try: + if "vc_config" in globals() and vc_config and vc_config.get_bool("KEYWORDS_FROM_VOICE_TYPE", False): + keywords = [] + for key in self.VC_TTS_voice_type: + keywords.append(key) + json_data["payload"]["inbound_attributes"]["cfg"]["keywords"] = keywords + except Exception: + pass + if self.dialog_attributes_provider != None: + json_data["payload"]["dialog_attributes"]["provider"] = self.dialog_attributes_provider + if self.commands_mode !=0 : + json_data["payload"]["commands"] = [] + if self.VC_mode == 0 :#and dev_type != "7PIN_BK_coze":#长对话模式打开vad + json_data["payload"]["inbound_attributes"]["disable_vad"] = False + # 新会话开始：在收到服务端 started 前，禁止发送音频帧（避免 session not initialized） + self._vc_session_started = 0 + try: + self.fsm.on_session_start_request() + except Exception: + pass + self.AI_client_VC_send(ujson.dumps(json_data)) + try: + _tid = json_data.get("header", {}).get("trace_id", "") + print("[VC][proto] Start enqueued trace_id={} VC_mode={} disable_vad={} ws_conn={}".format( + _tid or "-", + self.VC_mode, + json_data.get("payload", {}).get("inbound_attributes", {}).get("disable_vad", "?"), + self.AI_client_VC_isconnect)) + except Exception: + pass + def AI_client_VC_RequestToSpeak(self): + json_data = \ + { + "event": "cancel", + "payload": {} + } + print("AI_client_VC_RequestToSpeak!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") + # 模式切换等场景 play_tts(..., is_close_VC=1) 会置 Close_flag=1；用户再按 AI 键走 cancel→canceled→ + # uplink_start 时，FSM 与 _uplink_request_start 均会因 Close_flag 拒绝开麦，表现为“切回长对话后按键无上行”。 + try: + if self.VC_mode == 0: + self.Close_flag = 0 + self._vc_pending_interrupt_uplink = 1 + self._vc_clear_stale_record_state("RequestToSpeak") + except Exception: + pass + # 只做“打断+清播放队列”，不要在这里关音频队列线程（会导致后续 TTS/提示音不稳定，且触发竞态） + EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_init}) + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_send,"data":ujson.dumps(json_data)}) + utime.sleep_ms(1) + try: + self.fsm.reset_rx_phases() + except Exception: + pass + def AI_client_new_voice(self): #上传未读的语音地址/包括音频地址也可以 + json_data = \ + { + "event": "audio.convert", + "payload": { + "audio_convert_request": { + "audio_url": self.AI_new_voice_url, + "bit_depth": 16, + "channels": 1, + "sample_rate": 16000, + "target_format": "pcm" + } + } + } + self.AI_new_voice_url = "" + print("AI_client_new_voice!!!!!!!!!!!!!!!!!!!!!!!!!!!!!{}".format(json_data)) + self.AI_client_VC_send(ujson.dumps(json_data)) + def send_tone(self): + EventMesh.publish("audio_montage_play",["send_tone.mp3"]) diff --git a/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_session_media.py b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_session_media.py new file mode 100644 index 00000000..b5d2038c --- /dev/null +++ b/2.sdk/project/7PIN_1_5/sdk/services/microPython/ports/quectel/core/vc_session_media.py @@ -0,0 +1,1695 @@ +# -*- coding: utf-8 -*- +"""AI_VC mixin：ASR 上行/录音、TTS 下行、播放队列与本地采播。""" +from vc_session_imports import * # noqa: F401,F403 +import _thread + +class VCSessionMediaMixin: + def get_vc_tts_playback_active(self, *args, **kwargs): + try: + return 1 if self._vc_tts_playback_active() else 0 + except Exception: + return 0 + def get_vc_stream_active(self, *args, **kwargs): + # 供本地提示音侧查询：TTS 流播放期间本地不可抢占。 + try: + if self._vc_tts_playback_active(): + return 1 + except Exception: + pass + try: + if getattr(self.fsm, "await_played", False): + return 1 + except Exception: + pass + try: + if getattr(self.fsm, "tts_rx_phase", VC_TtsRxPhase.Idle) != VC_TtsRxPhase.Idle: + return 1 + except Exception: + pass + try: + t = getattr(self.fsm, "turn", VCTurnState.Idle) + if t in (VCTurnState.TtsReceiving, VCTurnState.Playing, VCTurnState.AwaitPlayedAck): + return 1 + except Exception: + pass + try: + if getattr(self, "_tts_pending", 0) == 1: + return 1 + except Exception: + pass + return 0 + def _vc_tts_pipeline_busy(self): + try: + if getattr(self.fsm, "await_played", False): + return True + if getattr(self.fsm, "tts_rx_phase", VC_TtsRxPhase.Idle) != VC_TtsRxPhase.Idle: + return True + t = getattr(self.fsm, "turn", VCTurnState.Idle) + if t in (VCTurnState.TtsReceiving, VCTurnState.Playing, VCTurnState.AwaitPlayedAck): + return True + if getattr(self, "_tts_pending", 0) == 1: + return True + if getattr(self, "VC_Play_flag", 0) == 1: + return True + if self._vc_local_play_busy(): + return True + except Exception: + pass + return False + def _vc_local_play_busy(self): + try: + if getattr(self, "_vc_local_play_active", 0) != 1: + return False + until = getattr(self, "_vc_local_play_until_ms", 0) + if until and utime.ticks_diff(until, utime.ticks_ms()) > 0: + return True + self._vc_local_play_active = 0 + self._vc_local_play_until_ms = 0 + except Exception: + pass + return False + def _vc_local_play(self, path, play_mode=1, loop=1, hold_ms=650): + """本地 MP3 短音：播前停录/停流式 TTS，避免与麦克风采集叠用。""" + try: + if getattr(self, "record_data_save_flag", 0) == 0 and getattr(self, "VC_record_state_flag", 0) != 0: + self._uplink_request_stop("local_play") + utime.sleep_ms(30) + except Exception: + pass + try: + if self._vc_tts_playback_active(): + self._ptt_hard_mute_local_playback() + try: + self.VC_Play_End() + except Exception: + pass + except Exception: + pass + try: + hold = int(hold_ms) + if hold < 200: + hold = 200 + self._vc_local_play_active = 1 + self._vc_local_play_until_ms = utime.ticks_add(utime.ticks_ms(), hold) + except Exception: + pass + try: + self.aud.play(play_mode, loop, path) + except Exception as e: + try: + self._vc_local_play_active = 0 + self._vc_local_play_until_ms = 0 + except Exception: + pass + usys.print_exception(e) + def _vc_tts_playback_active(self): + """仅服务端 TTS/PCM 流在播（VC_Play_Start）；aud.play 本地短音不算。""" + try: + if getattr(self, "VC_Play_flag", 0) == 1: + return True + except Exception: + pass + try: + if getattr(self, "play_pcm", None) is not None: + return True + except Exception: + pass + return False + def _vc_vad_uplink_stop_eligible(self): + # 对应旧 VC_state >= SpeechStart + try: + if getattr(self.fsm, "speech_uplink_open", 0): + return True + t = getattr(self.fsm, "turn", VCTurnState.Idle) + return t not in (VCTurnState.Idle, VCTurnState.Listening) + except Exception: + return False + def _vc_clear_stale_record_state(self, reason=""): + """record_state_flag 非 0 但可能无真实录音时先停录复位，避免 uplink 门控拒开麦。""" + try: + if getattr(self, "VC_record_state_flag", 0) != 0: + self.VC_Speech_End() + except Exception: + try: + self.VC_record_state_flag = 0 + except Exception: + pass + def _vc_effective_asr_text(self): + try: + t = (getattr(self, "AI_ASR_str", "") or "").strip() + if t: + return t + return (getattr(self, "_last_asr_result_text", "") or "").strip() + except Exception: + return "" + def _vc_long_dialogue_auto_uplink_ok(self, reason): + """长对话 canceled/completed 自动开麦：须有有效 ASR，且防抖/勿在 LLM 无 TTS 后抢开。""" + try: + if reason not in ("canceled", "completed", "local.play_end", "audio.played"): + return True + if getattr(self, "Close_flag", 0) == 1: + return False + if reason == "canceled" and getattr(self, "_vc_pending_interrupt_uplink", 0) == 1: + pass + elif not self._vc_effective_asr_text(): + k = getattr(self, "_last_tts_kind", "") or "" + if reason == "local.play_end" and k == "boot_prompt": + pass + else: + try: + print("[VC][uplink_ctrl] Deny START by {} cause=empty_asr".format(reason)) + except Exception: + pass + return False + if reason == "completed": + if getattr(self, "_turn_has_tts_started", 0) == 0: + try: + if getattr(self, "_llm_completed_ms", 0) or getattr(self.fsm, "turn", None) == VCTurnState.Thinking: + print("[VC][uplink_ctrl] Deny START by completed cause=llm_without_tts") + return False + except Exception: + pass + try: + now = utime.ticks_ms() + last = getattr(self, "_vc_last_auto_uplink_ms", 0) + if last and utime.ticks_diff(now, last) < 1200: + print("[VC][uplink_ctrl] Deny START by {} cause=auto_uplink_debounce".format(reason)) + return False + self._vc_last_auto_uplink_ms = now + except Exception: + pass + except Exception: + pass + return True + def _uplink_request_start(self, reason="", *args, **kwargs): + # 统一入口：避免多处同时 start 造成冲突（尤其是 TTS/播放阶段误开麦导致回声触发与状态紊乱） + try: + if self.Close_flag == 1: + try: + if reason in ("canceled", "local.play_end", "audio.played", "key.press/continue"): + print("[VC][uplink_ctrl] Deny START by {} cause=Close_flag".format(reason)) + except Exception: + pass + return + except Exception: + return + + # 连续对话才允许“自动开新一轮上行” + try: + if self.VC_mode != 0: + try: + if reason in ("canceled", "local.play_end", "audio.played"): + print("[VC][uplink_ctrl] Deny START by {} cause=VC_mode({})".format(reason, self.VC_mode)) + except Exception: + pass + return + except Exception: + return + + try: + if self.VC_mode == 0 and not self._vc_long_dialogue_auto_uplink_ok(reason): + return + except Exception: + pass + + try: + if not self._vc_capture_allowed(reason): + try: + print("[VC][uplink_ctrl] Deny START by {} cause=playback_busy".format(reason)) + except Exception: + pass + return + except Exception: + return + + # ===== 业务门控（收敛到 FSM turn/session）===== + try: + if getattr(self.fsm, "session", VCSessionState.Disconnected) != VCSessionState.Started: + try: + if reason in ("canceled", "local.play_end", "audio.played"): + print("[VC][uplink_ctrl] Deny START by {} cause=fsm.session".format(reason)) + except Exception: + pass + return + turn = getattr(self.fsm, "turn", VCTurnState.Idle) + # 仅 Listening/SpeechEnded 允许开麦进入下一轮；其它阶段一律拒绝（避免抢跑打乱链路） + if turn not in (VCTurnState.Listening, VCTurnState.SpeechEnded): + try: + if reason in ("canceled", "local.play_end", "audio.played"): + print("[VC][uplink_ctrl] Deny START by {} cause=fsm.turn({})".format(reason, turn)) + except Exception: + pass + return + except Exception: + return + + # 若当前不在录音上行，才允许开新一轮 + if getattr(self, "VC_record_state_flag", 0) != 0: + # canceled 打断后：可能是僵死 flag=2（started 未真开麦），先停录复位再开麦 + if reason == "canceled": + try: + self._vc_clear_stale_record_state("uplink_start/canceled") + except Exception: + pass + if getattr(self, "VC_record_state_flag", 0) != 0: + try: + if reason in ("canceled", "local.play_end", "audio.played"): + print("[VC][uplink_ctrl] Deny START by {} cause=VC_record_state_flag({})".format( + reason, getattr(self, "VC_record_state_flag", 0))) + except Exception: + pass + return + + # 新一轮开始录音：允许上行（上一轮可能已置 silence 闸门） + if getattr(self, "record_data_save_flag", 0) == 0: + self._server_vad_silence = 0 + + print("[VC][uplink_ctrl] Request START by {}".format(reason)) + try: + if reason == "canceled" and getattr(self, "_vc_pending_interrupt_uplink", 0) == 1: + self._vc_pending_interrupt_uplink = 0 + print("[VC][uplink_ctrl] interrupt_uplink armed -> start (canceled)") + except Exception: + pass + self.VC_Speech_Start_now() + def _uplink_request_stop(self, reason="", *args, **kwargs): + # 统一入口：避免多处同时 stop 造成冲突 + if getattr(self, "record_data_save_flag", 0) == 0: + self._server_vad_silence = 1 + + if getattr(self, "VC_record_state_flag", 0) == 0: + try: + if getattr(self, "VC_mode", 0) != 1: + print("[VC][long_idle] _uplink_request_stop skip (record_state==0) reason={}".format(reason)) + except Exception: + pass + return + + try: + if getattr(self, "VC_mode", 0) != 1: + print("[VC][long_idle] step=uplink_stop_exec reason={} VC_record_state_flag={}".format( + reason, getattr(self, "VC_record_state_flag", -1))) + except Exception: + pass + print("[VC][uplink_ctrl] Request STOP by {}".format(reason)) + self.VC_Speech_End() + def _uplink_mark_start(self, first_len=0): + try: + if self._uplink_active == 1: + return + self._uplink_active = 1 + self._uplink_bytes = 0 + self._uplink_start_ms = utime.ticks_ms() + print("[VC][uplink] START len={} {} VC_record_state_flag={}".format( + first_len, self._vc_fsm_log_snip(), self.VC_record_state_flag)) + except Exception: + pass + def _uplink_mark_end(self, reason=""): + try: + if self._uplink_active != 1: + return + dur = utime.ticks_diff(utime.ticks_ms(), self._uplink_start_ms) if self._uplink_start_ms else -1 + print("[VC][uplink] END reason={} bytes={} dur_ms={} {} VC_record_state_flag={}".format( + reason, self._uplink_bytes, dur, self._vc_fsm_log_snip(), self.VC_record_state_flag)) + except Exception: + pass + self._uplink_active = 0 + self._uplink_bytes = 0 + self._uplink_start_ms = 0 + + # 最近一次 TTS 的语义来源（dialog/prompt/mode_switch_long/mode_switch_ptt/...） + self._last_tts_kind = "" + # 下一个“无 dialog_id 的提示音 TTS”在 tts.started 时采用该 kind + self._pending_tts_kind = "" + def request_uplink_start(self, reason=""): + EventMesh.publish("publish_AI_btnInfo", {"func": self._uplink_request_start, "data": reason}) + def request_uplink_stop(self, reason=""): + EventMesh.publish("publish_AI_btnInfo", {"func": self._uplink_request_stop, "data": reason}) + def set_VC_stop_play_audio_flag(self,event = None, msg = None): + self.VC_stop_play_audio_flag = msg + print("self.VC_stop_play_audio_flag:{}".format(self.VC_stop_play_audio_flag)) + def rate_change(self,event,msg):#直接设置语速（数字） + self.speech_rate = msg + EventMesh.publish("set_storage_config",["speech_rate",self.speech_rate]) + print("set_speech_rate:{}".format(self.speech_rate)) + def rate_move(self,event,msg):#设置语速(模糊(å¿«/慢一点)) + if msg == 1 and self.speech_rate<50:#增大 + self.speech_rate += 10 + else:#减小 + if self.speech_rate >-50: + self.speech_rate -= 10 + EventMesh.publish("set_storage_config",["speech_rate",self.speech_rate]) + print("set_speech_rate:{}".format(self.speech_rate)) + def find_rate(self,event,msg):#查询APP语速 + print("查询语速处理：原始数据为：{}".format(self.speech_rate)) + y = 1 + if self.speech_rate >= -50 and self.speech_rate < 0:#负数 + y = 1 - (-0.01 * self.speech_rate) + elif self.speech_rate <= 50 and self.speech_rate >= 0:#正数 + y = 0.02 * self.speech_rate + 1.0 + print("处理后数据为{}".format(y)) + rate = str(y) + "倍" + return rate + def set_record_data_save_flag(self,event = None, msg = None): + self.record_data_save_flag = msg + def get_AI_audio_queue(self, event, msg): + return self.AI_audio_queue + def audio_cb(self,event): + if event == 0: + EventMesh.publish(constants.EVENT.SYSTEM_UP)#唤醒休眠 + self.audio_play_state =1 + #EventMesh.publish("set_audio_state", 1) + print('audio-play start.') + # 本地 aud.play（开录/思考短音等）不改动 AI 灯，仅 TTS 流由 VC_Play_Start 控绿灯 + if self._vc_led_state_debug_enabled(): + print("[LED] audio_cb start skip (local_play)") + return + elif event == 7: + self.audio_play_state = 0 + #EventMesh.publish("set_audio_state", 0) + print('audio-play finish.!!!!!!') + try: + if getattr(self, "VC_mode", -1) == 1 and getattr(self, "_ptt_await_server", 0) == 1: + self._vc_publish_led("thinking", "audio_cb/finish/ptt_await") + return + except Exception: + pass + try: + if getattr(self, "VC_record_state_flag", 0) != 0: + self._vc_publish_led("listening", "audio_cb/finish/recording") + return + except Exception: + pass + try: + if self._vc_tts_playback_active(): + if self._vc_led_state_debug_enabled(): + print("[LED] audio_cb finish skip (tts_playback)") + return + except Exception: + pass + try: + if getattr(self, "_ptt_led_lock_listening", 0) == 1: + return + except Exception: + pass + self._vc_led_sync_from_fsm() + def set_tts_volume(self,event,msg): + EventMesh.publish("set_storage_config",["VC_TTS_volume",msg])#音量配置 + self.VC_TTS_volume = msg + print("set_tts_volume:{} {}".format(self.VC_TTS_volume,msg)) + if int(msg) <= 10: #最小一定要设置1要不然没有声音 + volume = 10 //10 + self.aud.setVolume(volume) + else: + self.aud.setVolume(self.VC_TTS_volume //10) + pass + def get_audio_play_state(self, event=None, msg = None): + return self.audio_play_state + def set_VC_record_state_flag(self,data): + + self.VC_record_state_flag = data + print("[VC] record_state_flag={}".format(self.VC_record_state_flag)) + def VC_Speech_Start_now(self): + # 新一轮开始录音：允许上行（上一轮可能已置 silence 闸门） + self._server_vad_silence = 0 + self.VC_Speech_Start() + if self.VC_record_state_flag == 1: + self.VC_record_state_flag =2 + print("[VC] Speech_Start_now record_state={}".format(self.VC_record_state_flag)) + def VC_Speech_Start(self): + try: + + #self.VC_state = VC_State_SpeechStart + print("[VC] Speech_Start record_state={} AI_ASR_flag={}".format(self.VC_record_state_flag,self.AI_ASR_flag)) + if self.AI_ASR_flag != 0: + return + + if self.VC_record_state_flag == 0: + # 新一轮开始录音：允许上行（上一轮可能已置 silence 闸门） + self._server_vad_silence = 0 + self.AI_ASR_str = "" + self.AI_ASR_str_tmp = "" + self._last_asr_result_text = "" + try: + self._asr_completed_tone_done = 0 + except Exception: + pass + + self.VC_record_thread_close()#关闭录音线程 + _thread.stack_size(self.Audio_buf_size+8*1024) + self.VC_record_thread_id = _thread.start_new_thread(self.VC_record_thread,()) + _thread.stack_size(8*1024) + print("self.VC_record_thread_id:{}".format(self.VC_record_thread_id)) + try: + + if self.VC_record_TYPE == VC_record_TYPE_AMR : + try : + self.record.stream_start(audio.Record.AMRNB, self.VC_record_samplerate, 0) + except Exception as e: + usys.print_exception(e) + elif self.VC_record_TYPE == VC_record_TYPE_PCM: + print("record_pcm.stream_read_start()!!!!!!!!!!!!") + self.record.stream_start(audio.Record.WAV, self.VC_record_samplerate, 0) + # if self.record_pcm != None: + # try: + # self.record_pcm.stream_stop() + # self.record_pcm.close() + # self.record_pcm = None + # except Exception as e: + # usys.print_exception(e) + # if self.record_pcm == None: + # ret = self.record_pcm = audio.Audio.PCM(0,1,16000,0,1,2) + # print("audio.Audio.PCM:ret:{}".format(ret)) + # ret = self.record_pcm.stream_read_start() + # print("self.record_pcm.stream_read_start:ret:{}".format(ret)) + except Exception as e: + usys.print_exception(e) + + self.VC_record_state_flag = 1 + # 录音硬件已起：协议侧应对应 audio.create（PTT）或长对话直传 PCM + try: + print("[VC][proto] record_start sr={} pcm_type={} VC_mode={} session_started={} " + "ptt_audio_created={} thread_id={}".format( + self.VC_record_samplerate, self.VC_record_TYPE, self.VC_mode, + getattr(self, "_vc_session_started", -1), + getattr(self, "_ptt_audio_created", -1), + self.VC_record_thread_id)) + except Exception: + pass + # 录音开始：聆听灯（PTT 蓝快闪 / 长对话蓝慢闪） + try: + self._vc_publish_led("listening", "VC_Speech_Start") + except Exception: + pass + try: + if getattr(self, "VC_mode", -1) == 1: + self._ptt_on_record_hardware_started() + except Exception: + pass + + except Exception as e: + self.websocket_error() + self.VC_record_state_flag = 1 + self.VC_Speech_End()#关闭录音 + usys.print_exception(e) + def VC_Speech_End(self): + try: + self._uplink_mark_end("VC_Speech_End") + self.AI_ASR_flag = 0 + self.AI_ASR_str_tmp = "" + # 勿在此清空 AI_ASR_str：vad.silence 停麦常早于 asr.completed，清空会导致 check_text:len:0 连播 think 音 + #EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_asr}) + + if self.VC_record_state_flag == 0: + return + #self.AI_client_asr_StopTranscription_requset()#请求停止 + if self.VC_record_TYPE == VC_record_TYPE_AMR: + self.record.stream_stop() + self.record.stop()#关闭录音 + Record_read_buf = bytearray(self.Audio_buf_size) + for i in range(10): + Record_len = self.record.stream_read(Record_read_buf, self.Audio_buf_size) + # if VC_AUDIO_FRAME_LOG: + # print("Record_len:!!!!!!{}".format(Record_len)) + if Record_len <= 0: + break + elif self.VC_record_TYPE == VC_record_TYPE_PCM: + self.record.stream_stop() + self.record.stop()#关闭录音 + Record_read_buf = bytearray(self.Audio_buf_size) + for i in range(10): + Record_len = self.record.stream_read(Record_read_buf, self.Audio_buf_size) + # if VC_AUDIO_FRAME_LOG: + # print("Record_len:!!!!!!{}".format(Record_len)) + if Record_len <= 0: + break + # if self.record_pcm != None: + # try: + # self.record_pcm.stream_stop() + # self.record_pcm.close() + # self.record_pcm = None + # except Exception as e: + # usys.print_exception(e) + except Exception as e: + usys.print_exception(e) + self.VC_record_thread_close()#关闭录音线程 + if self.AI_record_tmp_queue.empty() != True: + self.AI_record_tmp_queue.clear() + + self.VC_record_state_flag = 0 + # PTT 一轮录音结束：清长按语义，避免后续短按松手误走 end_speech 而不播提示音 + try: + if getattr(self, "VC_mode", -1) == 1: + self.AI_key_long_press_flag = 0 + except Exception: + pass + # 录音停止：单对话进入思考态（等服务器），勿继续蓝闪聆听 + try: + if getattr(self, "VC_mode", -1) == 1: + self._ptt_await_server = 1 + self._vc_publish_led("thinking", "VC_Speech_End/ptt") + else: + self._ptt_await_server = 0 + self._vc_publish_led("idle", "VC_Speech_End") + except Exception: + pass + print("[VC] Speech_End record_state={}".format(self.VC_record_state_flag)) + def VC_record_thread_close(self): + if self.VC_record_thread_id != 0: + count =50 + while True: + count -= 1 + if self.VC_record_thread_lock == 1: + self.VC_record_thread_lock = 2 + if self.VC_record_thread_lock == 0 or count <=0: + break + else: + print("VC_record_thread_thread going:{}...".format(count)) + utime.sleep_ms(1) + self.VC_record_thread_lock = 0 + print("VC_record_thread_thread_close") + if self.VC_record_thread_id != 0: + try: + _thread.stop_thread(self.VC_record_thread_id) + except Exception as e: + usys.print_exception(e) + self.VC_record_thread_id = 0 + def VC_record_thread(self): + ##return + print("[VC] record_thread start") + count = 0 + RequestToSpeak_flag = 0 + # try: + # uos.remove('/usr/test.pcm') + # except Exception as e: + # usys.print_exception(e) + #record_data_queue = Queue(100) + #fd = open('/usr/test.pcm','w') + # if self.VC_record_TYPE == 1: + # h_data= [0x23,0x21 ,0x41 ,0x4D ,0x52 ,0x0A ] + # h_data_b = bytes(h_data) + # self.AI_client_VC_send(h_data_b) + AI_record_tmp_queue_len = 0 + vad_buf = bytearray() + VAD_data_count = 0 + vad_state = 0 + self.VAD_count = 0 + while True: + try: + if self.VC_record_thread_lock == 2: + break + self.VC_record_thread_lock = 1 + # 死循环接收数据 + #if self.VC_state == VC_State_SpeechStart: + Record_read_buf = bytearray(self.Audio_buf_size) + + Record_len = 0 + if self.VC_record_TYPE == VC_record_TYPE_PCM:#pcm + Record_len = self.record.stream_read(Record_read_buf, self.Audio_buf_size) + #utime.sleep_ms(60) + + else : + + Record_len = self.record.stream_read(Record_read_buf, self.Audio_buf_size) + utime.sleep_ms(600) + + #Record_len = 1024*2 + #print("VC_record_thread:{}".format(Record_len)) + if Record_len > 0: + read_buf = bytes(Record_read_buf[:Record_len]) + # if VC_AUDIO_FRAME_LOG: + # print("Record_len:{}".format(Record_len)) + if self.record_data_save_flag == 1: + if self.record_fd == None: + try: + uos.remove('/usr/record_data.pcm') + except Exception as e: + pass + # 二进制写入，避免文本模式导致数据异常 + self.record_fd = open('/usr/record_data.pcm', 'wb') + self.record_fd_len = 0 + if self.record_fd != None: + try: + self.record_fd.write(read_buf) + print("write record:{}".format(len(read_buf))) + self.record_fd_len = self.record_fd_len+len(read_buf) + if self.record_fd_len >=500*1024: + EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) + except Exception as e: + usys.print_exception(e) + EventMesh.publish("publish_AI_btnInfo",{"func":self.Close}) + # count+=Record_len + # if count >150*1024: + # self.record_fd.close() + #self.record_fd = None + #VAD检测 + if self.is_VAD == 1: + vad_state = 0 + tmp_c = 1000 + while True: + tmp_c -=1 + if tmp_c <=0: + break + #vad_buf_len = len(vad_buf) + if VAD_data_count == 0:#没数据的时候 + vad_buf = Record_read_buf[:self.VAD_data_len] + VAD_data_count = self.VAD_data_len + self.vad.custom_vad_reset()#重置vad + else:#有数据的时候 + if Record_len - VAD_data_count < self.VAD_data_len :#剩余长度不够一次填充，先保留，但不填算法 + vad_buf = Record_read_buf[VAD_data_count:Record_len] + VAD_data_count = 0 + break + else:#剩余长度够填一次数据 + vad_buf = Record_read_buf[VAD_data_count:VAD_data_count+self.VAD_data_len] + VAD_data_count += len(vad_buf) + #print("vad_buf_len:{},{}".format(len(vad_buf),VAD_data_count)) + + if len(vad_buf) >= self.VAD_data_len: + vad_state = self.vad.custom_vad_process(vad_buf,len(vad_buf))#给vad数据 + #print("vad_state:{}".format(vad_state)) + + if VAD_data_count >= Record_len: + VAD_data_count = 0 + break + if vad_state == 1: + print("检测到VAD=1！！！！！！！！！！！！！") + + # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_RequestToSpeak})#打断 + # RequestToSpeak_flag = 1 + + self.VAD_count+=1 + if self.VAD_count >= 20: + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_VC_RequestToSpeak})#打断 + + else: + if self.VAD_count != 0: + EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Play_Pause,"data":0})#恢复音频播放 + self.VAD_count = 0 + if self._vc_tts_pipeline_busy(): + pass + if vad_state == 1: + self.AI_record_tmp_queue.put(read_buf)#入缓存队列 + AI_record_tmp_queue_len += len(read_buf) + + + else: + VAD_data_count = 0 + if self.VC_record_state_flag <=1 :#没发送前先存起来 + self.AI_record_tmp_queue.put(read_buf) + AI_record_tmp_queue_len += len(read_buf) + else: + AI_record_tmp_queue_len = 0 + while True: + if self.AI_record_tmp_queue.empty() == True: + break + else: + pass + # 协议门控：会话未 started 前禁止上行音频（否则 session not initialized） + if VC_PROTO_GUARDS and self._vc_session_started != 1 and self.record_data_save_flag == 0: + _ = self.AI_record_tmp_queue.get() + AI_record_tmp_queue_len = max(0, AI_record_tmp_queue_len - len(_)) + continue + if not (self._server_vad_silence == 1 and self.record_data_save_flag == 0): + self.AI_client_VC_send(self.AI_record_tmp_queue.get()) #发送之前存的数据 + else: + self._uplink_mark_end("silence_gate(drop_queue)") + _ = self.AI_record_tmp_queue.get() + # 当前帧发送前协议门控（可用 VC_PROTO_GUARDS 临时关闭）： + # - session started 前禁止发帧 + # - Push-To-Talk 在 audio.create 前禁止发帧 + if VC_PROTO_GUARDS and ( + (self._vc_session_started != 1 and self.record_data_save_flag == 0) or + (self.VC_mode == 1 and self._ptt_audio_created != 1 and self.record_data_save_flag == 0) + ): + # 未 ready：转入缓存，等待 started 再发（并做容量控制） + try: + self.AI_record_tmp_queue.put(read_buf) + AI_record_tmp_queue_len += len(read_buf) + while AI_record_tmp_queue_len > 128*1024 and self.AI_record_tmp_queue.empty() != True: + d = self.AI_record_tmp_queue.get() + AI_record_tmp_queue_len = max(0, AI_record_tmp_queue_len - len(d)) + except Exception: + pass + else: + if not (self._server_vad_silence == 1 and self.record_data_save_flag == 0): + self.AI_client_VC_send(read_buf) + else: + self._uplink_mark_end("silence_gate(drop_buf)") + + + + + + + #print("VC_record_thread:{}".format(len(read_buf))) + #count+=len(read_buf) + # if count>=500*1024:#结束 + # self.AI_client_VC_end_speech() + # break + # count+=1 + # if count %10 == 0: + # print("VC_record_thread on line {} asr recv :{} send:{}".format(self.VC_record_state_flag,_thread.threadIsRunning(self.AI_client_asr_recv_id),_thread.threadIsRunning(self.AI_VC_send_data_queue_thread_id))) + + # if self.VC_state >= VC_State_SpeechEnd: + # break + + except Exception as e: + usys.print_exception(e) + #break + self.VC_record_thread_lock = 0 + if self.record_data_save_flag == 1: + utime.sleep_ms(1) + else: + utime.sleep_ms(100) + #fd.close() + print("VC_record_thread:exit") + self.VC_record_thread_id = 0#线程退出 + self.VC_record_thread_lock = 0 + def voice_play_10S(self,event,msg):#按键开启留言播放等待10S定时器判断 + self.voice_timer.start(1000,1,self.voice_judge) + def voice_judge(self, *args, **kwargs): + EventMesh.publish("publish_AI_btnInfo",{"func":self.voice_judger_cb}) + def voice_judger_cb(self): + self.voice_cnt += 1 + # print("语音留言定时器cnt = {}".format(self.voice_cnt)) + if self.voicerupt_flag == 1:#按键被按下 + self.pwk_notice_flag = 1#表示进入收听未听留言 + self.AI_VC_start() + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 + self.voice_cnt = 0 + # self.recv_voice = 0#进入留言定时器，留言清空 + print("进入留言定时器1，留言清空！！！！！") + EventMesh.publish("update_play_new_voice_info")#更新已读状态 + self.voice_timer.stop() + if self.pwk_cnt_voice > 0: + EventMesh.publish("get_new_voice_msg_S") + if self.voice_cnt > 15:#10S后退出播放,5S是为了给语音提示留时间 + self.voice_cnt = 0 + self.recv_voice = 0#进入留言定时器，留言清空 + if self.voice_choose_flag == 1:#播放完语音后，有冲突的时候关闭 + self.voice_choose_flag = 0 + print("进入留言定时器2，留言清空！！！！！") + self.voice_timer.stop() + def revoice_judge(self, *args, **kwargs): + EventMesh.publish("publish_AI_btnInfo",{"func":self.revoice_judger_cb}) + def revoice_judger_cb(self): + self.revoice_cnt += 1 + # print("语音留言回复定时器cnt = {}".format(self.revoice_cnt)) + if self.revoice_cnt > 20:#10S后退出播放,10S是为了给语音提示留时间 + self.revoice_cnt = 0 + self.voice_flag=0 + EventMesh.publish("set_1M_nowplay_flag",0) + print("语音留言回复停止") + self.revoice_timer.stop() + def music_play(self, *args, **kwargs): + EventMesh.publish("publish_AI_btnInfo",{"func":self.music_play_cb}) + def music_play_cb(self): + self.music_cnt += 1 + if self.music_cnt > 4:#4S后播放音乐,4S是为了给语音提示留时间 + self.AI_VC_start() + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 + self.music_cnt = 0 + self.music_timer.stop() + def history_play(self, *args, **kwargs): + EventMesh.publish("publish_AI_btnInfo",{"func":self.history_play_cb}) + def history_play_cb(self): + self.his_cnt += 1 + if self.his_cnt > 4:#4S后播放,4S是为了给语音提示留时间 + self.AI_VC_start() + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 + self.his_cnt = 0 + self.his_timer.stop() + def wait_play(self, *args, **kwargs): + EventMesh.publish("publish_AI_btnInfo",{"func":self.wait_play_cb}) + def wait_play_cb(self): + self.wait_cnt += 1 + if self.wait_voicerupt_flag == 1:#按键被按下 + EventMesh.publish("set_is_listen_wait",1)#已听 + self.pwk_notice_flag = 1#表示进入收听未听留言 + self.AI_VC_start() + self.wait_sort = 1 + self.AI_TTS_str = "第"+str(self.wait_sort)+"条" + # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 + # self.wait_cnt = 0 + # print("进入留言定时器1，留言清空！！！！！") + # EventMesh.publish("update_play_new_voice_info")#更新已读状态 + self.wait_timer.stop() + if self.wait_cnt > 10:#10S后退出播放,5S是为了给语音提示留时间 + self.wait_cnt = 0 + self.recv_voice = 0#进入留言定时器，留言清空 + EventMesh.publish("set_history_voice_flag",0) + EventMesh.publish("set_wait_voice_flag",0) + EventMesh.publish("set_open_device",0)#开机不听 + self.wait_first_flag = 0 + if self.voice_choose_flag == 1:#播放完语音后，有冲突的时候关闭 + self.voice_choose_flag = 0 + print("进入留言定时器2，留言清空！！！！！") + self.wait_timer.stop() + def wait_single_play(self, *args, **kwargs): + EventMesh.publish("publish_AI_btnInfo",{"func":self.wait_single_play_cb}) + def wait_single_play_cb(self): + self.wait_cnt += 1 + if self.wait_cnt > 2:#2S后播放,2S是为了给语音提示留时间 + self.AI_VC_start() + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 + self.wait_cnt = 0 + self.wait_timer.stop() + def put_record_data(self,data): + try: + if self._ptt_should_drop_inbound_tts(): + return + except Exception: + pass + #print("下发音频数据1:{}".format(len(data[0]))) + ret=self.AI_audio_queue.put(data) + try: + if self.tts_in_data_len == 0 and data and len(data[0]) > 0: + try: + sid = getattr(self, "_cur_sid", "") + did = getattr(self, "_cur_dialog_id", "") + except Exception: + sid, did = "", "" + print("[TTS] first_chunk len={} play_pcm={} VC_Play_flag={} sid={} did={}".format( + len(data[0]), + 1 if self.play_pcm != None else 0, + getattr(self, "VC_Play_flag", 0), + sid, + did, + )) + except Exception: + pass + self.tts_in_data_len += len(data[0]) + def play_tts(self,event = None,msg= None): + EventMesh.publish("publish_AI_btnInfo",{"func":self.play_tts_cb,"data":msg}) + def play_tts_cb(self,data): + try: + tts_str = data[0] + is_close_VC = data[1] + if tts_str != "": + self.AI_TTS_str = tts_str + if is_close_VC != None: + self.Close_flag = is_close_VC + if self.AI_VC_send_data_queue.empty()!=True: + self.AI_VC_send_data_queue.clear() + if self.AI_ASR_send_data_queue.empty()!=True: + self.AI_ASR_send_data_queue.clear() + try: + if is_close_VC: + EventMesh.publish("set_led_state", {"ai_state": "idle"}) + else: + EventMesh.publish("set_led_state", {"ai_state": "thinking"}) + except Exception: + pass + self.AI_ASR_send_buf_len = 0 + self._uplink_request_stop("play_tts_cb") + self.websocket_deconnect_VC() + self.AI_VC_start() + + # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + except Exception as e: + usys.print_exception(e) + pass + def VC_Play_Start(self): + try: + if self._ptt_should_skip_remote_playback(): + if self._vc_led_state_debug_enabled(): + print("[VC] VC_Play_Start skip (ptt_interrupt)") + return + except Exception: + pass + self.tts_in_data_len = 0 + self.tts_out_data_len = 0 + self._tts_first_write_ms = 0 + try: + self._tts_completed_wall_ms = 0 + except Exception: + pass + if self.VC_Play_flag == 0: + try: + _ptt_fast_open = False + try: + if getattr(self, "VC_mode", -1) == 1: + _ptt_fast_open = True + except Exception: + pass + # 本地提示音/文件播放可能占用同一音频通道，导致 TTS 流播放“写入成功但无声”； + # 在开启 TTS PCM 前先停掉本地播放，避免资源冲突。 + try: + self.aud.stopPlayStream() + self.aud.stopAll() + except Exception: + pass + # 资源互斥：播放前确保录音上行已停止（无 AEC 场景） + try: + if self.record_data_save_flag == 0 and self.VC_record_state_flag != 0: + self._uplink_request_stop("VC_Play_Start") + utime.sleep_ms(30) + except Exception: + pass + if self.VC_play_TYPE == VC_play_TYPE_PCM: + if self.play_pcm == None: + # 刚停采集时延后打开播音 PCM，且失败时重试，降低「pcm open write only fail」 + last_err = None + _open_retries = 1 if _ptt_fast_open else 3 + for i in range(_open_retries): + try: + if not _ptt_fast_open: + utime.sleep_ms(80 + i * 60) + self.play_pcm = audio.Audio.PCM(0, 1, self.VC_play_samplerate, 1, 1, 2) + ret = self.play_pcm.stream_write_start() + print("self.record_pcm.stream_write_start:ret:{}".format(ret)) + last_err = None + break + except Exception as e: + last_err = e + try: + if self.play_pcm != None: + self.play_pcm.close() + except Exception: + pass + self.play_pcm = None + if last_err: + raise last_err + elif self.VC_play_TYPE == VC_play_TYPE_MP3: + pass + self.VC_Play_flag = 1 + try: + if not self._ptt_should_skip_remote_playback(): + self._vc_publish_led("playing", "VC_Play_Start/tts") + except Exception: + pass + except Exception as e: + self.VC_Play_End() + usys.print_exception(e) + def VC_Play_Pause(self,Pause_flag): + try: + if self.VC_play_TYPE == VC_play_TYPE_PCM: + if self.play_pcm != None: + self.play_pcm.stream_pause(Pause_flag) + except Exception as e: + self.VC_Play_End() + usys.print_exception(e) + def VC_Play_End(self): + self.VC_Play_flag = 0 + if self.VC_play_TYPE == VC_play_TYPE_PCM: + if self.VC_play_samplerate == 8000: + self.aud.stopPlayStream() + self.aud.stopAll() + elif self.VC_play_samplerate == 16000: + if self.play_pcm != None: + print("Close:play_pcm.close()!!!!!!!!!!!!!!!!!!") + try: + self.play_pcm.stream_stop() + self.play_pcm.close() + except Exception as e: + usys.print_exception(e) + self.play_pcm=None + + self.aud.stopPlayStream() + self.aud.stopAll() + elif self.VC_play_TYPE == VC_play_TYPE_MP3: + self.aud.stopPlayStream() + self.aud.stopAll() + if self.AI_audio_queue.empty()!=True: + self.AI_audio_queue.clear() + + # TTS 播完：按真实 FSM/录音态恢复灯；本地短音不走此路径 + try: + if getattr(self, "_vc_play_end_suppress_idle_led", 0): + if self._vc_led_state_debug_enabled(): + print("[LED] VC_Play_End skip led (suppress)") + elif getattr(self, "VC_record_state_flag", 0) != 0: + self._vc_publish_led("listening", "VC_Play_End/recording") + else: + self._vc_led_sync_from_fsm() + except Exception: + pass + # 关键日志：定位“tts 结束绿灯仍不灭”究竟是 base 态（如 charging/full）还是 playing 未退出 + try: + if "vc_config" in globals() and vc_config and ( + vc_config.get_bool("LED_DEBUG", False) or vc_config.get_bool("KEY_LED_DEBUG", False) or vc_config.get_bool("HW_STATUS_DEBUG", False) + ): + try: + led = EventMesh.publish("get_led_debug_state") + except Exception: + led = None + print("[VC] VC_Play_End led={}".format(led)) + except Exception: + pass + def _vc_tts_bytes_per_sec(self): + try: + return int(self.VC_play_samplerate) * 2 + except Exception: + return 32000 + + # audio_audio.c: ring_buf_len = 64*1024；stream_get_buf_len 返回「空闲可写」而非待播 + _PCM_STREAM_RING_BYTES = 65536 + + def _vc_pcm_stream_pending_bytes(self, pcm_free): + if pcm_free < 0: + return -1 + if pcm_free >= self._PCM_STREAM_RING_BYTES: + return 0 + return self._PCM_STREAM_RING_BYTES - pcm_free + + def _vc_try_play_end_tick(self): + """tts.completed 后判定本地 PCM 是否真正播完，避免过早 VC_Play_End 截断 TTS。""" + try: + if not getattr(self.fsm, "await_played", False): + return False + in_len = getattr(self, "tts_in_data_len", 0) + out_len = getattr(self, "tts_out_data_len", 0) + try: + q_empty = self.AI_audio_queue.empty() + except Exception: + q_empty = True + tts_done_ms = getattr(self, "_tts_completed_wall_ms", 0) or getattr( + self.fsm, "last_tts_completed_ms", 0) + if not (in_len > 0 and out_len >= in_len and q_empty and tts_done_ms): + return False + + is_pcm_empty = (self.play_pcm is None) or ( + self.play_pcm is not None and self.play_pcm.stream_is_empty() == 1 + ) + pcm_free = 0 + if self.play_pcm is not None: + try: + pcm_free = self.play_pcm.stream_get_buf_len() + except Exception: + pcm_free = -1 + pcm_pending = self._vc_pcm_stream_pending_bytes(pcm_free) + if pcm_pending < 0: + pcm_pending = max(0, in_len - out_len) + + now = utime.ticks_ms() + bps = self._vc_tts_bytes_per_sec() + fw = getattr(self, "_tts_first_write_ms", 0) + # 丢弃明显陈旧的 completed 时间（上轮残留或 last_tts_completed_ms==1 等） + try: + if fw and tts_done_ms and utime.ticks_diff(tts_done_ms, fw) < 0: + tts_done_ms = now + if tts_done_ms and utime.ticks_diff(now, tts_done_ms) > 120000: + return False + except Exception: + pass + + try: + since_done = utime.ticks_diff(now, tts_done_ms) + except Exception: + since_done = 0 + debounce_ms = 150 + if pcm_pending > 0 and bps > 0: + tail_ms = max(debounce_ms, int(pcm_pending * 1000 // bps) + 200) + else: + tail_ms = debounce_ms + tail_ok = since_done >= tail_ms + # 与旧 ai_VC LocalRespondingEnded 一致：stream 空且环形缓冲无待播数据 + pcm_drained = is_pcm_empty and pcm_pending == 0 + ready = pcm_drained and tail_ok + + if not ready: + try: + if not getattr(self, "_tts_play_check_log_ms", 0) or utime.ticks_diff( + now, self._tts_play_check_log_ms + ) >= 800: + self._tts_play_check_log_ms = now + print( + "[TTS] play_end_wait since_done={}ms tail~{}ms " + "in={} out={} q_empty={} pcm_free={} pcm_pending={} pcm_empty={}".format( + since_done, tail_ms, in_len, out_len, + 1 if q_empty else 0, pcm_free, pcm_pending, + 1 if is_pcm_empty else 0, + ) + ) + except Exception: + pass + return False + + try: + print( + "[TTS] play_end_fire since_done={}ms tail~{}ms " + "in={} out={} pcm_free={} pcm_pending={} pcm_empty={}".format( + since_done, tail_ms, in_len, out_len, pcm_free, pcm_pending, + 1 if is_pcm_empty else 0, + ) + ) + except Exception: + pass + EventMesh.publish("publish_AI_btnInfo", {"func": self.VC_Play_End}) + self._fsm_apply_actions(self.fsm.on_local_play_end()) + try: + self.fsm.reset_rx_phases() + except Exception: + pass + self._tts_first_write_ms = 0 + return True + except Exception as e: + usys.print_exception(e) + return False + def AI_client_VC_send_speech(self): + json_data = \ + { + "header": { + "event": "audio.create", + + }, + } + + if self.VC_mode == 1:#单对话模式 + # 协议门控（可用 VC_PROTO_GUARDS 临时关闭） + if (not VC_PROTO_GUARDS) or (self._vc_session_started == 1): + self._ptt_audio_created = 1 + try: + print("[VC][proto] audio.create send session_started={} proto_guards={}".format( + self._vc_session_started, VC_PROTO_GUARDS)) + except Exception: + pass + self.AI_client_VC_send(ujson.dumps(json_data)) + else: + try: + print("[VC][proto] audio.create SKIP session_started={} VC_PROTO_GUARDS={} " + "(等对端 started 后再开麦)".format(self._vc_session_started, VC_PROTO_GUARDS)) + except Exception: + pass + + + + + + + try: + self.fsm.on_server_asr_started() + except Exception: + pass + def AI_client_VC_end_speech(self): + json_data = \ + { + "header": { + "event": "audio.completed", + + }, + } + self.AI_client_VC_send(ujson.dumps(json_data)) + self._ptt_audio_created = 0 + EventMesh.publish("publish_AI_btnInfo",{"func":self._uplink_request_stop, "data":"audio.completed"}) + def AI_client_VC_synthesis(self): + # 幂等保护：在上一条 synthesis 还未等到 tts.started 前，不重复发（防止多兜底/多入口重复触发） + try: + if self._tts_pending == 1: + return + except Exception: + pass + json_data = \ + { + "event": "synthesis", + "payload": { + "text": self.AI_TTS_str + } + } + # 语义标记只用于“是否自动开麦”的策略。 + # 当前策略：长对话不自动开麦（需按键开始），因此这里不对提示音做自动开麦标记。 + self.AI_TTS_str = "" + # 标记为“等待 tts.started”，避免在 synthesis→tts.started 窗口误开下一轮录音上行 + self._tts_pending = 1 + try: + self._tts_pending_ms = utime.ticks_ms() + except Exception: + self._tts_pending_ms = 0 + try: + self.fsm.on_local_synthesis_sent() + except Exception: + pass + print("AI_client_VC_synthesis!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") + # if self.VC_state == VC_State_RequestToRespond: + # print("AI_client_VC_RequestToRespond 重复 tts_str:{}\nasr_str:{}".format(self.AI_TTS_str,self.AI_ASR_str)) + self.AI_client_VC_send(ujson.dumps(json_data)) + def AI_client_VC_audio_played(self): + # V2.5：播完上报；与 VC_PROTO_GUARDS 无关（见文件头说明） + try: + if self.AI_client_VC_isconnect != 1 or self.AI_client_VC == None: + return + json_data = {"event": "audio.played", "payload": {}} + print("AI_client_VC_audio_played") + self.AI_client_VC_send(ujson.dumps(json_data)) + except Exception as e: + usys.print_exception(e) + def set_vad_silence(self,event = None,msg= None): + self.vad_silence = msg + EventMesh.publish("set_storage_config",["vad_silence",self.vad_silence]) + def set_VC_TTS_voice_type(self,event = None,msg= None): + self.VC_TTS_voice_type["APP_voice"] = msg + #self.VC_TTS_voice_type[self.VC_TTS_voice_key] + self.VC_TTS_voice_key = "APP_voice" + EventMesh.publish("set_storage_config",["VC_TTS_voice_type",self.VC_TTS_voice_type]) + EventMesh.publish("set_storage_config",["VC_TTS_voice_key",self.VC_TTS_voice_key]) + print("[VC] TTS_voice key={} map={}".format(self.VC_TTS_voice_key, self.VC_TTS_voice_type)) + def change_volume(self,event =None, msg =None): + if msg == 0 : + self.VC_TTS_volume -= 10 + if self.VC_TTS_volume < 0: + self.VC_TTS_volume = 0 + return 0 + else: + self.aud.setVolume(self.VC_TTS_volume //10) + EventMesh.publish("set_storage_config",["VC_TTS_volume",self.VC_TTS_volume]) + EventMesh.publish("post_device_state") + EventMesh.publish("adjust_volume_toBle",self.VC_TTS_volume) + return 1 + elif msg == 1: + self.VC_TTS_volume += 10 + if self.VC_TTS_volume > 100: + self.VC_TTS_volume = 100 + return 0 + else: + self.aud.setVolume(self.VC_TTS_volume //10) + EventMesh.publish("set_storage_config",["VC_TTS_volume",self.VC_TTS_volume]) + EventMesh.publish("post_device_state") + EventMesh.publish("adjust_volume_toBle",self.VC_TTS_volume) + return 1 + elif msg >= 100:#修改为具体音量 + if msg > 200: + self.VC_TTS_volume = 100 + return 0 + else: + self.VC_TTS_volume = msg - 100#还原数据 + self.aud.setVolume(self.VC_TTS_volume //10) + EventMesh.publish("set_storage_config",["VC_TTS_volume",self.VC_TTS_volume]) + EventMesh.publish("post_device_state") + EventMesh.publish("adjust_volume_toBle",self.VC_TTS_volume) + return 1 + def NLP_set_vad_silence(self,event,msg): + if msg == 1:#聆听变快 + self.vad_silence -= 1000 + if self.vad_silence < 1000: + AI_TTS_str = "已经是最快了" + self.vad_silence = 1000 + else: + AI_TTS_str = "配置成功，当前语音断句速度是{}秒".format(self.vad_silence/1000) + elif msg == 0:#聆听变慢 + self.vad_silence += 1000 + if self.vad_silence > 4000: + self.vad_silence = 4000 + #EventMesh.publish("audio_montage_play",["yinliangyizuixiao.mp3"]) + #self.aud.play(1,0,'U:/static/yinliangyizuixiao.mp3')#音量已最小 + AI_TTS_str = "已经是最慢了" + else: + AI_TTS_str = "配置成功，当前语音断句速度是{}秒".format(self.vad_silence/1000) + EventMesh.publish("set_storage_config",["vad_silence",self.vad_silence]) + return AI_TTS_str + def change_voice(self,event ,msg): + voice = msg + # if "男声" in msg or "普通话" in msg: + # self.VC_TTS_voice_key = "男" + # else: + # self.VC_TTS_voice_key = voice + self.outbound_attributess_provider = "xun-fei" + if "男声" in msg or "普通话" in msg: + self.VC_TTS_voice_key = "男" + self.outbound_attributess_provider = "volcengine" + elif "女声" in msg: + self.VC_TTS_voice_key = "女" + self.outbound_attributess_provider = "volcengine" + elif "童声" in msg: + self.VC_TTS_voice_key = "ç«¥" + self.outbound_attributess_provider = "volcengine" + elif "香港粤语" in msg: + self.VC_TTS_voice_key = "香港粤语" + elif "四川话" in msg: + self.VC_TTS_voice_key = "四川" + elif "上海话" in msg: + self.VC_TTS_voice_key = "上海" + elif "安徽合肥话" in msg: + self.VC_TTS_voice_key = "安徽合肥" + elif "山东话" in msg: + self.VC_TTS_voice_key = "山东话" + elif "陕西话" in msg: + self.VC_TTS_voice_key = "陕西" + elif "广东话" in msg: + self.VC_TTS_voice_key = "广东" + elif "成都话" in msg: + self.VC_TTS_voice_key = "成都" + elif "湖南话" in msg: + self.VC_TTS_voice_key = "湖南" + elif "苏州话" in msg: + self.VC_TTS_voice_key = "苏州" + elif "东北话" in msg: + self.VC_TTS_voice_key = "东北" + elif "河南话" in msg: + self.VC_TTS_voice_key = "河南" + elif "湖北话" in msg: + self.VC_TTS_voice_key = "湖北" + elif "闽南话" in msg: + self.VC_TTS_voice_key = "闽南" + elif "内蒙古话" in msg: + self.VC_TTS_voice_key = "内蒙古话" + EventMesh.publish("set_storage_config",["VC_TTS_voice_key",self.VC_TTS_voice_key]) + tts_play_str = "声音已改为{}".format(voice) + EventMesh.publish("play_tts",[tts_play_str,0]) + def play_single_voice(self,event = None,msg= None): + EventMesh.publish("publish_AI_btnInfo",{"func":self.play_single_voice_cb,"data":msg}) + def play_single_voice_cb(self,data): + try: + new_voice_url = data[0] #音频地址 + is_close_VC = data[1] + if new_voice_url != "": + self.AI_new_voice_url = new_voice_url + self.alarm_play_flag = 1 + if is_close_VC != None: + self.Close_flag = is_close_VC + if self.AI_VC_send_data_queue.empty()!=True: + self.AI_VC_send_data_queue.clear() + if self.AI_ASR_send_data_queue.empty()!=True: + self.AI_ASR_send_data_queue.clear() + EventMesh.publish("set_led_state", {"ai_state": "thinking"}) + self.AI_ASR_send_buf_len = 0 + self._uplink_request_stop("play_single_voice_cb") + self.websocket_deconnect_VC() + self.AI_VC_start() + self.play_new_voice_flag = 1 + # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + except Exception as e: + usys.print_exception(e) + pass + def play_new_voice(self,event = None,msg= None): + EventMesh.publish("publish_AI_btnInfo",{"func":self.play_new_voice_cb,"data":msg}) + def play_new_voice_cb(self,data): + try: + new_voice_url = data[0] #音频地址 + is_close_VC = data[1] + if new_voice_url != "": + self.recv_voice = 1#收到留言 + print("已收到最新留言！！！！！") + # EventMesh.publish("set_motor_time",[1,1,5,8,4])#设置马达参数 + self.AI_new_voice_url = new_voice_url + # if self.pwk_cnt_voice == 0 and self.pwk_notice_flag == 0:#并且不在多条留言的状态 + # EventMesh.publish("play_tts",["您有一条语音留言，10秒内按下中间键可以收听",1]) + if self.pwk_notice_flag == 0 and EventMesh.publish("get_app_new_voice_flag"):#不处于播放状态并且是小程序下发的语音而不是请求的 + self.pwk_cnt_voice += 1#加上当前这条 + EventMesh.publish("set_storage_config",["is_noread_voice",1]) + EventMesh.publish("set_app_new_voice_flag",0) + print("voice_add_to = {}".format(self.pwk_cnt_voice)) + print("self.pwk_notice_flag = {}".format(self.pwk_notice_flag)) + if self.pwk_cnt_voice >1: + if self.pwk_notice_flag == 0: + tts_str = "您有" + str(self.pwk_cnt_voice) + "条未读语音留言，10秒内按下中间键可以收听" + EventMesh.publish("play_tts",[tts_str,1]) + elif self.pwk_cnt_voice == 1: + if self.pwk_notice_flag == 0: + tts_str = "您有1条语音留言，10秒内按下中间键可以收听" + EventMesh.publish("play_tts",[tts_str,1]) + if is_close_VC != None: + self.Close_flag = is_close_VC + if self.AI_VC_send_data_queue.empty()!=True: + self.AI_VC_send_data_queue.clear() + if self.AI_ASR_send_data_queue.empty()!=True: + self.AI_ASR_send_data_queue.clear() + EventMesh.publish("set_led_state", {"ai_state": "thinking"}) + self.AI_ASR_send_buf_len = 0 + self._uplink_request_stop("play_new_voice_cb") + self.websocket_deconnect_VC() + self.AI_VC_start() + self.play_new_voice_flag = 1 + # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + except Exception as e: + usys.print_exception(e) + pass + def play_his_voice(self,event = None,msg= None): + EventMesh.publish("publish_AI_btnInfo",{"func":self.play_his_voice_cb,"data":msg}) + def play_his_voice_cb(self,data): + try: + new_voice_url = data[0] #音频地址 + is_close_VC = data[1] + time = data[2] + tts_play_str = time + "收到" + EventMesh.publish("play_tts",[tts_play_str,1]) + if new_voice_url != "": + self.recv_voice = 1#收到留言 + print("历史留言查看") + # tts_play_str = "收到" + # EventMesh.publish("play_tts",[tts_play_str,1]) + # EventMesh.publish("set_motor_time",[1,1,5,8,4])#设置马达参数 + self.AI_new_voice_url = new_voice_url + if is_close_VC != None: + self.Close_flag = is_close_VC + if self.AI_VC_send_data_queue.empty()!=True: + self.AI_VC_send_data_queue.clear() + if self.AI_ASR_send_data_queue.empty()!=True: + self.AI_ASR_send_data_queue.clear() + EventMesh.publish("set_led_state", {"ai_state": "thinking"}) + self.AI_ASR_send_buf_len = 0 + self._uplink_request_stop("play_his_voice_cb") + self.websocket_deconnect_VC() + self.AI_VC_start() + self.play_new_voice_flag = 1 + # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + except Exception as e: + usys.print_exception(e) + pass + def play_wait_voice(self,event = None,msg= None): + EventMesh.publish("publish_AI_btnInfo",{"func":self.play_wait_voice_cb,"data":msg}) + def play_wait_voice_cb(self,data): + try: + new_voice_url = data[0] #音频地址 + is_close_VC = data[1] + print("是否为第一次：{}".format(self.wait_first_flag)) + if self.wait_first_flag == 0: + tts_play_str = "您有" + str(len(EventMesh.publish("get_play_new_voice_list"))) + "条未听留言，10秒内按下中间键可以收听" + EventMesh.publish("play_tts",[tts_play_str,1]) + # self.wait_first_timer.start(1000,1,self.noread_wait_vocie) + self.wait_first_flag = 1 + elif self.wait_first_flag == 1: + self.wait_sort += 1 + self.AI_TTS_str = "第"+str(self.wait_sort)+"条" + if new_voice_url != "": + self.recv_voice = 1#收到留言 + print("未读留言查看") + self.AI_new_voice_url = new_voice_url + if is_close_VC != None: + self.Close_flag = is_close_VC + if self.AI_VC_send_data_queue.empty()!=True: + self.AI_VC_send_data_queue.clear() + if self.AI_ASR_send_data_queue.empty()!=True: + self.AI_ASR_send_data_queue.clear() + EventMesh.publish("set_led_state", {"ai_state": "thinking"}) + self.AI_ASR_send_buf_len = 0 + self._uplink_request_stop("play_wait_voice_cb") + self.websocket_deconnect_VC() + self.AI_VC_start() + self.play_new_voice_flag = 1 + # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + except Exception as e: + usys.print_exception(e) + pass + def noread_wait_vocie(self,*args):#如果一直没读 + self.wait_first_cnt += 1 + print("未读留言等待收听中：{}".format(self.wait_first_cnt)) + if EventMesh.publish("get_recv_voice") == 1:#如果检测到留言播放 + self.wait_first_cnt = 0 + self.wait_first_timer.stop() + if self.wait_first_cnt == 15: + EventMesh.publish("set_wait_first_flag",0) + EventMesh.publish("set_history_voice_flag",0) + EventMesh.publish("set_wait_voice_flag",0) + self.wait_first_cnt = 0 + self.wait_first_timer.stop() + def play_music(self,event = None,msg= None): + EventMesh.publish("publish_AI_btnInfo",{"func":self.play_music_cb,"data":msg}) + def play_music_cb(self,data): + try: + music_name = data[0] + new_voice_url = data[1] #音频地址 + is_close_VC = data[2] + if new_voice_url != "": + self.AI_new_voice_url = new_voice_url + self.music_play_flag = 1#表示音乐播放开启 + EventMesh.publish("play_tts",["下面为您播放歌曲{}".format(music_name),1]) + if is_close_VC != None: + self.Close_flag = is_close_VC + if self.AI_VC_send_data_queue.empty()!=True: + self.AI_VC_send_data_queue.clear() + if self.AI_ASR_send_data_queue.empty()!=True: + self.AI_ASR_send_data_queue.clear() + EventMesh.publish("set_led_state", {"ai_state": "thinking"}) + self.AI_ASR_send_buf_len = 0 + self._uplink_request_stop("play_music_cb") + self.websocket_deconnect_VC() + self.AI_VC_start() + self.play_new_voice_flag = 1 + # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + except Exception as e: + usys.print_exception(e) + def play_trans_tts(self,event = None,msg= None): + EventMesh.publish("publish_AI_btnInfo",{"func":self.play_trans_tts_cb,"data":msg}) + def play_trans_tts_cb(self,data): + try: + new_voice_url = data #音频地址 + if new_voice_url != "": + self.AI_new_voice_url = new_voice_url + if self.AI_VC_send_data_queue.empty()!=True: + self.AI_VC_send_data_queue.clear() + if self.AI_ASR_send_data_queue.empty()!=True: + self.AI_ASR_send_data_queue.clear() + EventMesh.publish("set_led_state", {"ai_state": "thinking"}) + self.AI_ASR_send_buf_len = 0 + self._uplink_request_stop("play_trans_tts_cb") + self.websocket_deconnect_VC() + self.AI_VC_start() + EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_client_new_voice})#播放留言 + self.play_new_voice_flag = 1 + # EventMesh.publish("publish_AI_btnInfo",{"func":self.VC_Speech_End}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.websocket_deconnect_VC}) + # EventMesh.publish("publish_AI_btnInfo",{"func":self.AI_VC_start}) + except Exception as e: + usys.print_exception(e) + def AI_audio_queue_thread_close(self): + print("AI_audio_queue_thread_close1!!!!!!!!!!!!!!!!!!!!!!") + if self.AI_audio_queue_thread_id != 0: + count =500 + while True: + count -= 1 + if self.AI_audio_queue_thread_lock == 0 or count <=0: + break + else: + self.AI_audio_queue_thread_lock = 2 + print("AI_audio_queue_thread going:{}...".format(count)) + utime.sleep_ms(1) + self.AI_audio_queue_thread_lock = 0 + print("AI_audio_queue_thread_close2!!!!!!!!!!!!!!!!!") + if self.AI_audio_queue_thread_id != 0: + try: + _thread.stop_thread(self.AI_audio_queue_thread_id) + except Exception as e: + usys.print_exception(e) + self.AI_audio_queue_thread_id = 0 + def AI_audio_queue_thread(self): + print("AI_audio_queue_thread start") + count = 0 + if self.AI_audio_queue.empty()!=True: + self.AI_audio_queue.clear() + print("音频数据还有剩余！！！！！！！！！！！！") + while True: + try: + if self.AI_audio_queue_thread_lock == 2 or self.VC_stop_play_audio_flag == 0: + print("AI_audio_queue_thread 该退出了") + break + # + count+=1 + if count%500 == 1 and VC_HEARTBEAT_LOG: + print("AI_audio_queue_thread on") + utime.sleep_ms(1) + #print("AI_audio_queue_thread") + if self.AI_audio_queue.empty() : + self.AI_audio_queue_thread_lock = 0 + + + else: + self.AI_audio_queue_thread_lock = 1 + data = self.AI_audio_queue.get() + self.close_timerout = 0 + if self.play_flag == 0: + self.play_flag =1 + #self.AI_client_tts_data_len=len(data) + #print("audio_queue:len:{}".format(len(data[0]))) + + if len(data[0]) > 10: + try: + if self._ptt_should_drop_inbound_tts(): + continue + except Exception: + pass + for i in range(5): + if self.VC_Play_flag == 0: + self.VC_Play_Start() + utime.sleep_ms(2) + else: + break + # 打断后 PCM 偶发未打开：加重试，避免「队列有数据、play_pcm=None、无声」 + if self.VC_play_TYPE == VC_play_TYPE_PCM and self.play_pcm is None and self.VC_stop_play_audio_flag == 1: + for j in range(4): + if self.VC_Play_flag == 0: + self.VC_Play_Start() + utime.sleep_ms(30 + j * 35) + if self.play_pcm is not None: + break + try: + if self.VC_play_TYPE == VC_play_TYPE_PCM: + # if self.VC_play_samplerate == 8000: + # tmp= self.aud.playStream(1,data[0] ) + # else: + write_buf = bytearray(data[0]) + #print("play_pcm.stream_write1 len:{}".format(len(write_buf))) + if self.play_pcm != None: + #print("play_pcm.stream_write !!!") + ret = self.play_pcm.stream_write(write_buf,len(write_buf)) + #ret= self.aud.playStream(1, data[0])#播放音频流 + if self.tts_out_data_len == 0: + try: + self._tts_first_write_ms = utime.ticks_ms() + except Exception: + self._tts_first_write_ms = 0 + try: + sid = getattr(self, "_cur_sid", "") + did = getattr(self, "_cur_dialog_id", "") + except Exception: + sid, did = "", "" + print("[TTS] first_write ret={} len={} sid={} did={}".format( + ret, len(write_buf), sid, did + )) + if ret <= 0: + print("[TTS] stream_write ret={} len={} (no audio?)".format(ret, len(write_buf))) + self.tts_out_data_len += ret + try: + if getattr(self.fsm, "await_played", False) and self.AI_audio_queue.empty(): + self._vc_try_play_end_tick() + except Exception: + pass + #print("play_pcm.stream_write2:{}".format(self.tts_out_data_len)) + elif self.VC_play_TYPE == VC_play_TYPE_MP3: + tmp= self.aud.playStream(3, data[0])#播放音频流 + + if tmp != 0: + print("2playStream:error tmp:{} data len:{}!!!!!!!!!!!!".format(tmp,len(data[0]))) + # # #播放失败的时候填充WAV头 + # data_len = len(data[0]) + # data_len_b = data_len.to_bytes(4,'little')#数据长度转16进制数组 + # h_data = [ 0x52 ,0x49 ,0x46 ,0x46 #,0x24 ,0x00 ,0x00 ,0x00 #文件标识、文件大小 + # ,0x57 ,0x41 ,0x56 ,0x45 ,0x66 ,0x6D ,0x74 ,0x20 #文件格式、格式块标识符 + # ,0x10 ,0x00 ,0x00 ,0x00 ,0x01 ,0x00 ,0x01 ,0x00 ,0x80 ,0x3E ,0x00 ,0x00 #格式块大小4、音频格式2、声道数2、采样率4 + # ,0x80 ,0x3E ,0x00 ,0x00 ,0x02 ,0x00 ,0x10 ,0x00 ,0x64 ,0x61 ,0x74 ,0x61]#每秒字节数4、数据块对齐2、位深度2、data4 + # h_data_b = bytes(h_data) + # file_len = data_len+len(h_data_b)+4+4 + # file_len_b = file_len.to_bytes(4,'little')#文件长度转16进制数组 + # h_data_b = h_data_b[:4]+file_len_b+h_data_b[4:]+data_len_b + # #self.aud.playStream(2, h_data_b)#播放音频流 + # tmp= self.aud.playStream(2, h_data_b+data[0])#播放音频流 + # print("3playStream:tmp:{} !!!!!!!!!!!!".format(tmp)) + + + mem = gc.mem_free() + #print('1剩余内存RAM:{} KB'.format(mem / 1024)) + except Exception as e: + print("AI_audio_queue_thread:ERROR:") + usys.print_exception(e) + + # if self.VC_state == VC_State_TTSEnd and self.AI_audio_queue.empty() :#self.AI_audio_queue为空且tts下发结束 + # #通知服务器播放结束 + # self.AI_client_VC_LocalRespondingEnded() + except Exception as e: + print("AI_audio_queue_thread:ERROR:") + usys.print_exception(e) + self.AI_audio_queue_thread_lock = 0 + self.AI_audio_queue_thread_id = 0 + print("AI_audio_queue_thread:exit")