admin/STT_Server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[项目打包]完成docker的打包，项目README更新，使用uv控制依赖。

Browse Source
This commit is contained in:
Ziyang.Zhang 2025-07-09 11:08:43 +08:00
parent db811763d4
commit 2172e58fce
13 changed files with 3888 additions and 131 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
Dockerfile
View File

@ -1,27 +0,0 @@
FROM python:3.9-slim
# 安装系统依赖
RUN apt-get update && apt-get install -y \
build-essential \
libsndfile1 \
ffmpeg \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
# 安装Python依赖
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# 复制应用代码
COPY . .
# 设置环境变量
ENV PYTHONPATH=/app
ENV PYTHONUNBUFFERED=1
# 暴露WebSocket端口
EXPOSE 10095
# 启动服务
CMD ["python", "src/server.py"]

59
README.md
View File

@ -17,7 +17,12 @@
```
.
├── main.py # 应用程序主入口，使用 uvicorn 启动服务
├── WEBSOCKET_API.md # WebSocket API 详细使用文档和示例
├── docker
│ ├── Dockerfile # Docker 镜像构建文件
│ ├── docker-compose.yml # Docker 容器编排文件
├── docs
│ ├── WEBSOCKET_API.md # WebSocket API 详细使用文档和示例
│ ├── SystemArchitecture.md # 系统架构文档
├── src
│ ├── server.py # FastAPI 应用核心，管理生命周期和全局资源
│ ├── runner
@ -30,12 +35,17 @@
│ │ ├── endpoint
│ │ │ └── asr_endpoint.py # WebSocket 的业务逻辑端点
│ │ └── router.py # WebSocket 路由
│ └── ...
└── tests
├── runner
│ └── asr_runner_test.py # ASRRunner 的单元测试 (异步)
└── websocket
└── websocket_asr.py # WebSocket 服务的端到端测试
│ ├── core
│ │ └── model_loader.py # 模型加载器
│ ├── utils
│ │ └── logger.py # 日志记录器
│ │ └── data_format.py # 数据格式转换
│ │ └── mock_websocket.py # 模拟 WebSocket 客户端
├── tests
│ ├── runner
│ │ └── asr_runner_test.py # ASRRunner 的单元测试 (异步)
│ ├── websocket
│ │ └── websocket_asr.py # WebSocket 服务的端到端测试
```
## 🚀 快速开始
@ -95,3 +105,38 @@ python tests/websocket/websocket_asr.py
python test_main.py
```
## 🐳 使用 Docker 部署
### 1. 构建 Docker 镜像
在项目根目录下，运行：
```bash
docker build -t asr-server:x.x.x -f docker/Dockerfile .
```
### 2. 运行 Docker 容器
```bash
docker run -d -p 11096:11096 --name asr-server -v ~/.cache/modelscope:/root/.cache/modelscope asr-server:x.x.x
```
### 环境变量说明
- `SPEAKERS_URL`: 说话人数据库 API 的 URL。
示例/默认 SPEAKERS_URL="http://172.23.30.120:11200/api/v1/speakers/"
此url为后端api提供的查询所有数据库中说话人信息的接口
- `LOG_LEVEL`: 日志等级。
示例/默认 LOG_LEVEL="INFO"
`LOG_LEVEL` 为总项目日志等级，
- `LOG_LEVEL_ASR_SERVER`: 日志等级。
示例/默认 LOG_LEVEL_ASR_SERVER="INFO"
`LOG_LEVEL_ASR_SERVER` 为 ASR 服务日志等级，优先级高于 `LOG_LEVEL`

23
docker-compose.yml
View File

@ -1,23 +0,0 @@
version: '3.8'
services:
funasr:
build: .
container_name: funasr-websocket
volumes:
- .:/app
# 如果需要使用本地模型缓存
- ~/.cache/modelscope:/root/.cache/modelscope
ports:
- "10095:10095"
environment:
- PYTHONUNBUFFERED=1
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
restart: unless-stopped
command: python src/server.py --device cuda --ngpu 1

36
docker/Dockerfile Normal file
View File

@ -0,0 +1,36 @@
FROM python:3.10-slim
# 更换系统源
# 科大大的 Debian 12 (bookworm) 镜像源
RUN echo "deb http://mirrors.ustc.edu.cn/debian bookworm main contrib non-free" > /etc/apt/sources.list && \
echo "deb http://mirrors.ustc.edu.cn/debian bookworm-updates main contrib non-free" >> /etc/apt/sources.list && \
echo "deb http://mirrors.ustc.edu.cn/debian-security bookworm-security main" >> /etc/apt/sources.list && \
echo "deb http://mirrors.ustc.edu.cn/debian bookworm-backports main contrib non-free" >> /etc/apt/sources.list
# 安装系统依赖
RUN apt-get update && apt-get install -y \
build-essential \
libsndfile1 \
ffmpeg \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
# 安装Python依赖
COPY ../requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
# 复制应用代码
COPY ../src/ /app/src/
COPY ../logs/ /app/logs/
COPY ../main.py /app/
# 设置环境变量
ENV SPEAKERS_URL="http://172.23.30.120:11200/api/v1/speakers/" \
LOG_LEVEL="INFO" \
LOG_LEVEL_ASR_SERVER="INFO"
# 暴露WebSocket端口
EXPOSE 11096
# 启动服务
CMD ["python", "main.py"]

94
docs/SystemArchitecture.md Normal file
View File

@ -0,0 +1,94 @@
# 系统架构
本项目是一个基于 FunASR 和 FastAPI 构建的高性能、实时的语音识别（ASR）WebSocket 服务。其核心架构设计旨在处理实时的流式音频数据，并通过 "一发多收" 的广播模式，将识别结果分发给多个客户端。
## 核心组件
系统主要由以下几个核心组件构成，它们各司其职，通过异步和多线程协作，实现了高效的实时语音处理：
1. **WebSocket 服务 (FastAPI)**
- **文件**: `src/websockets/`, `src/server.py`, `main.py`
- **职责**: 作为系统的网络入口，负责处理 WebSocket 连接。它使用 FastAPI 构建，提供异步的、非阻塞的 I/O 处理能力，能够高效地管理大量并发客户端连接。`asr_endpoint.py` 是核心端点，负责根据客户端声明的 `mode` (sender/receiver) 将连接路由到 `ASRRunner`。
2. **会话管理器 (ASRRunner)**
- **文件**: `src/runner/ASRRunner.py`
- **职责**: 这是整个系统的"大脑"和协调中心。它管理所有活跃的语音识别会话（`SenderAndReceiver`，简称 SAR）。
- **会话生命周期**: 当一个 `sender` 连接时，`ASRRunner` 会创建一个新的 SAR 实例；当 `receiver` 连接时，会将其加入到指定的 SAR 中。
- **异步桥梁**: `ASRRunner` 运行在主 `asyncio` 事件循环中，负责从 `sender` 的 WebSocket 连接异步接收音频数据，然后通过线程安全的队列 (`queue.Queue`) 将数据传递给同步的 `ASRPipeline`。同时，它也负责接收来自 Pipeline 的最终结果，并将其异步广播给所有 `receiver`。
3. **语音处理流水线 (ASRPipeline)**
- **文件**: `src/pipeline/ASRpipeline.py`
- **职责**: 这是实际执行语音处理任务的核心引擎。每个 SAR 会话都拥有一个独立的 `ASRPipeline` 实例，该实例在自己的后台线程中运行。
- **模块化设计**: Pipeline 内部由多个 `Functor` (如 VAD, ASR, SPK) 组成，通过一系列内部队列连接，形成一个处理链。
- **处理流程**:
1. **VAD (Voice Activity Detection)**: 检测音频流中的有效语音片段。
2. **ASR (Automatic Speech Recognition)**: 将语音片段转换为文字。
3. **SPK (Speaker Recognition)**: 识别说话人（声纹识别）。
4. **ResultBinder**: 将 ASR 的文本结果和 SPK 的说话人结果合并，生成最终的识别消息。
4. **原子操作单元 (Functor)**
- **文件**: `src/functor/`
- **职责**: `Functor` 是 Pipeline 中执行具体原子任务的单元。每个 Functor 都是一个独立的类，负责调用底层 FunASR 模型来执行 VAD、ASR 或 SPK 等任务。这种设计使得处理流程更加清晰和模块化。
## 流程图
下面是系统处理一次完整语音识别请求的流程图，展示了从客户端连接到收到识别结果的全过程。
```mermaid
sequenceDiagram
participant Sender Client
participant Receiver Client
participant FastAPI WebSocket Endpoint
participant ASRRunner
participant ASRPipeline (Thread)
participant Functors (VAD, ASR, SPK)
par
Sender Client->>+FastAPI WebSocket Endpoint: 发起连接 (mode=sender, session_id=S1)
FastAPI WebSocket Endpoint->>+ASRRunner: new_SAR(ws, name="S1")
ASRRunner->>ASRRunner: 创建 SenderAndReceiver (SAR) 实例
ASRRunner->>ASRPipeline (Thread): 创建并运行 Pipeline 实例
ASRPipeline (Thread)->>Functors (VAD, ASR, SPK): 初始化 Functor 线程
ASRRunner->>-FastAPI WebSocket Endpoint: 返回成功
FastAPI WebSocket Endpoint->>-Sender Client: 连接建立
and
Receiver Client->>+FastAPI WebSocket Endpoint: 发起连接 (mode=receiver, session_id=S1)
FastAPI WebSocket Endpoint->>+ASRRunner: join_SAR(ws, name="S1")
ASRRunner->>ASRRunner: 将 Receiver 加入 S1 的接收者列表
ASRRunner->>-FastAPI WebSocket Endpoint: 返回成功
FastAPI WebSocket Endpoint->>-Receiver Client: 连接建立
end
loop 音频流传输与处理
Sender Client->>ASRRunner: 发送音频数据块
ASRRunner->>ASRPipeline (Thread): (via Queue) 传递音频数据
ASRPipeline (Thread)->>Functors (VAD, ASR, SPK): (via sub-queues) 分发数据
Note over Functors (VAD, ASR, SPK): 1. VAD检测语音<br/>2. ASR识别文本<br/>3. SPK识别说话人<br/>4. ResultBinder合并结果
Functors (VAD, ASR, SPK)->>ASRPipeline (Thread): (via callback) 返回最终识别结果
ASRPipeline (Thread)->>ASRRunner: (via thread-safe callback) 发送结果
end
ASRRunner->>ASRRunner: 收到结果，准备广播
ASRRunner-->>Sender Client: 广播识别结果
ASRRunner-->>Receiver Client: 广播识别结果
par
Sender Client->>FastAPI WebSocket Endpoint: 关闭连接
FastAPI WebSocket Endpoint->>ASRRunner: (触发异常)
ASRRunner->>ASRPipeline (Thread): 发送停止信号
ASRPipeline (Thread)->>Functors (VAD, ASR, SPK): 停止 Functor 线程
Note right of ASRRunner: 清理会话资源
and
Receiver Client->>FastAPI WebSocket Endpoint: 关闭连接
FastAPI WebSocket Endpoint->>ASRRunner: (触发异常)
ASRRunner->>ASRRunner: 从接收者列表移除
end
```
## 架构优势
- **高并发和低延迟**: 采用 `asyncio` 和 WebSocket，网络层能够处理大量并发连接。音频处理在独立的线程中进行，避免了 CPU 密集型任务阻塞事件循环，保证了低延迟。
- **解耦与模块化**: `WebSocket Endpoint`、`ASRRunner` 和 `ASRPipeline` 职责清晰，相互解耦。`Functor` 的设计使得添加或修改处理步骤变得容易。
- **鲁棒性**: 每个识别会话（SAR）都是隔离的，一个会话的失败不会影响其他会话。优雅的关闭逻辑确保了资源的正确释放。
- **可扩展性**: "一发多收" 的广播模式可以轻松扩展到大量 `receiver`，适用于多种实时应用场景。

5
main.py
View File

@ -4,7 +4,10 @@ from src.utils.logger import get_module_logger, setup_root_logger
from datetime import datetime
time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
setup_root_logger(level="DEBUG", log_file=f"logs/main_{time}.log")
# 日志等级 LOG_LEVEL_ASR_SERVER > LOG_LEVEL > INFO(default)
logger_level = os.getenv("LOG_LEVEL", "INFO")
logger_level = os.getenv("LOG_LEVEL_ASR_SERVER", logger_level)
setup_root_logger(level=logger_level, log_file=f"logs/main_{time}.log")
logger = get_module_logger(__name__)
if __name__ == "__main__":

32
paerser.py
View File

@ -1,32 +0,0 @@
def pre_remove_details(input_string: str) -> str:
start_tag = '</details>'
start = input_string.find(start_tag, 0)
if start == -1:
return input_string, "unfind %s" % start_tag
return input_string[start + len(start_tag):], "success remove %s" % start_tag
def pre_remove_markdown(input_string: str) -> str:
start_tag = '```markdown'
end_tag = '```'
start = input_string.find(start_tag, 0)
if start == -1:
return input_string, "unfind %s" % start_tag
end = input_string.find(end_tag, start + 11)
if end == -1:
return input_string, "unfind %s" % end_tag
return input_string[start + 11:end].strip(), "success remove %s" % start_tag
def main(input_string: str) -> dict:
result = input_string
statuses = []
result, detail_status = pre_remove_details(result)
statuses.append(detail_status)
result, markdown_status = pre_remove_markdown(result)
statuses.append(markdown_status)
return {
"result": result,
"status": statuses
}

20
pyproject.toml Normal file
View File

@ -0,0 +1,20 @@
[project]
name = "asr-server"
version = "0.1.0"
requires-python = ">=3.9.21"
dependencies = [
"addict==2.4.0",
"datasets==2.21.0",
"fastapi==0.115.14",
"funasr==1.2.6",
"numpy==2.0.1",
"pillow==11.1.0",
"pydantic==2.11.3",
"pydub>=0.25.1",
"simplejson==3.20.1",
"sortedcontainers==2.4.0",
"torch==2.3.1",
"torchaudio==2.3.1",
"uvicorn==0.35.0",
"websockets==12.0",
]

330
requirements.txt
View File

@ -1,11 +1,335 @@
# This file was autogenerated by uv via the following command:
# uv pip compile pyproject.toml -o requirements.txt
addict==2.4.0
# via asr-server (pyproject.toml)
aiohappyeyeballs==2.6.1
# via aiohttp
aiohttp==3.12.13
# via
# datasets
# fsspec
aiosignal==1.4.0
# via aiohttp
aliyun-python-sdk-core==2.16.0
# via
# aliyun-python-sdk-kms
# oss2
aliyun-python-sdk-kms==2.16.5
# via oss2
annotated-types==0.7.0
# via pydantic
antlr4-python3-runtime==4.9.3
# via
# hydra-core
# omegaconf
anyio==4.9.0
# via starlette
async-timeout==5.0.1
# via aiohttp
attrs==25.3.0
# via aiohttp
audioread==3.0.1
# via librosa
certifi==2025.6.15
# via requests
cffi==1.17.1
# via
# cryptography
# soundfile
charset-normalizer==3.4.2
# via requests
click==8.2.1
# via uvicorn
crcmod==1.7
# via oss2
cryptography==45.0.5
# via aliyun-python-sdk-core
datasets==2.21.0
# via asr-server (pyproject.toml)
decorator==5.2.1
# via librosa
dill==0.3.8
# via
# datasets
# multiprocess
editdistance==0.8.1
# via funasr
exceptiongroup==1.3.0
# via anyio
fastapi==0.115.14
# via asr-server (pyproject.toml)
filelock==3.18.0
# via
# datasets
# huggingface-hub
# torch
# triton
frozenlist==1.7.0
# via
# aiohttp
# aiosignal
fsspec==2024.6.1
# via
# datasets
# huggingface-hub
# torch
funasr==1.2.6
# via asr-server (pyproject.toml)
h11==0.16.0
# via uvicorn
hf-xet==1.1.5
# via huggingface-hub
huggingface-hub==0.33.2
# via datasets
hydra-core==1.3.2
# via funasr
idna==3.10
# via
# anyio
# requests
# yarl
jaconv==0.4.0
# via funasr
jamo==0.4.1
# via funasr
jieba==0.42.1
# via funasr
jinja2==3.1.6
# via torch
jmespath==0.10.0
# via aliyun-python-sdk-core
joblib==1.5.1
# via
# librosa
# pynndescent
# scikit-learn
kaldiio==2.18.1
# via funasr
lazy-loader==0.4
# via librosa
librosa==0.11.0
# via funasr
llvmlite==0.44.0
# via
# numba
# pynndescent
markupsafe==3.0.2
# via jinja2
modelscope==1.27.1
# via funasr
mpmath==1.3.0
# via sympy
msgpack==1.1.1
# via librosa
multidict==6.6.3
# via
# aiohttp
# yarl
multiprocess==0.70.16
# via datasets
networkx==3.4.2
# via torch
numba==0.61.2
# via
# librosa
# pynndescent
# umap-learn
numpy==2.0.1
pyaudio==0.2.14
pydantic==2.11.7
# via
# asr-server (pyproject.toml)
# datasets
# kaldiio
# librosa
# numba
# pandas
# pytorch-wpe
# scikit-learn
# scipy
# soundfile
# soxr
# tensorboardx
# torch-complex
# umap-learn
nvidia-cublas-cu12==12.1.3.1
# via
# nvidia-cudnn-cu12
# nvidia-cusolver-cu12
# torch
nvidia-cuda-cupti-cu12==12.1.105
# via torch
nvidia-cuda-nvrtc-cu12==12.1.105
# via torch
nvidia-cuda-runtime-cu12==12.1.105
# via torch
nvidia-cudnn-cu12==8.9.2.26
# via torch
nvidia-cufft-cu12==11.0.2.54
# via torch
nvidia-curand-cu12==10.3.2.106
# via torch
nvidia-cusolver-cu12==11.4.5.107
# via torch
nvidia-cusparse-cu12==12.1.0.106
# via
# nvidia-cusolver-cu12
# torch
nvidia-nccl-cu12==2.20.5
# via torch
nvidia-nvjitlink-cu12==12.9.86
# via
# nvidia-cusolver-cu12
# nvidia-cusparse-cu12
nvidia-nvtx-cu12==12.1.105
# via torch
omegaconf==2.3.0
# via hydra-core
oss2==2.19.1
# via funasr
packaging==25.0
# via
# datasets
# huggingface-hub
# hydra-core
# lazy-loader
# pooch
# tensorboardx
# torch-complex
pandas==2.3.0
# via datasets
pillow==11.1.0
# via asr-server (pyproject.toml)
platformdirs==4.3.8
# via pooch
pooch==1.8.2
# via librosa
propcache==0.3.2
# via
# aiohttp
# yarl
protobuf==6.31.1
# via tensorboardx
pyarrow==20.0.0
# via datasets
pycparser==2.22
# via cffi
pycryptodome==3.23.0
# via oss2
pydantic==2.11.3
# via
# asr-server (pyproject.toml)
# fastapi
pydantic-core==2.33.1
# via pydantic
pydub==0.25.1
pytest==8.3.5
# via asr-server (pyproject.toml)
pynndescent==0.5.13
# via umap-learn
python-dateutil==2.9.0.post0
# via pandas
pytorch-wpe==0.0.1
# via funasr
pytz==2025.2
# via pandas
pyyaml==6.0.2
# via
# datasets
# funasr
# huggingface-hub
# omegaconf
requests==2.32.4
# via
# datasets
# funasr
# huggingface-hub
# modelscope
# oss2
# pooch
scikit-learn==1.7.0
# via
# librosa
# pynndescent
# umap-learn
scipy==1.15.3
# via
# funasr
# librosa
# pynndescent
# scikit-learn
# umap-learn
sentencepiece==0.2.0
# via funasr
setuptools==80.9.0
# via modelscope
simplejson==3.20.1
# via asr-server (pyproject.toml)
six==1.17.0
# via
# oss2
# python-dateutil
sniffio==1.3.1
# via anyio
sortedcontainers==2.4.0
# via asr-server (pyproject.toml)
soundfile==0.13.1
# via
# funasr
# librosa
soxr==0.5.0.post1
# via librosa
starlette==0.46.2
# via fastapi
sympy==1.14.0
# via torch
tensorboardx==2.6.4
# via funasr
threadpoolctl==3.6.0
# via scikit-learn
torch==2.3.1
# via
# asr-server (pyproject.toml)
# torchaudio
torch-complex==0.4.4
# via funasr
torchaudio==2.3.1
# via asr-server (pyproject.toml)
tqdm==4.67.1
# via
# datasets
# funasr
# huggingface-hub
# modelscope
# umap-learn
triton==2.3.1
# via torch
typing-extensions==4.14.1
# via
# aiosignal
# anyio
# exceptiongroup
# fastapi
# huggingface-hub
# librosa
# multidict
# pydantic
# pydantic-core
# torch
# typing-inspection
# uvicorn
typing-inspection==0.4.1
# via pydantic
tzdata==2025.2
# via pandas
umap-learn==0.5.9.post2
# via funasr
urllib3==2.5.0
# via
# modelscope
# requests
uvicorn==0.35.0
# via asr-server (pyproject.toml)
websockets==12.0
# via asr-server (pyproject.toml)
xxhash==3.5.0
# via datasets
yarl==1.20.1
# via aiohttp

11
requirements.txt.backup
View File

@ -1,11 +0,0 @@
pytest==7.3.1
pytest-cov==4.1.0
flake8==6.0.0
black==23.3.0
isort==5.12.0
flask==2.3.2
requests==2.31.0
websockets==11.0.3
numpy==1.24.3
funasr==0.10.0
modelscope==1.9.5

1
src/functor/base.py
View File

@ -167,6 +167,7 @@ class FunctorFactory:
spk_functor = SPKFunctor(sv_pipeline=models["spk"])
spk_functor.set_audio_config(audio_config)
# spk_functor.set_model(model)
spk_functor.bake()
logger.debug(f"创建spk functor[完成]")
return spk_functor

47
src/functor/spk_functor.py
View File

File diff suppressed because one or more lines are too long

3332
uv.lock generated Normal file
View File

File diff suppressed because it is too large Load Diff