bisheng系列(三)- 本地部署(后端 1.2版本)
一、导读
环境:Ubuntu 24.04、open Euler 23.03、Windows 11、WSL 2、Python 3.10 、bisheng 1.2.0
背景:需要bisheng二开商用,故而此处进行本地部署,便于后期调试开发
时间:20250612
说明:bisheng的1.2版本部署与1.1.1有些许不同,故另开一章
注意:本节基于下面章节的内容,如有疑问,请阅读:
bisheng系列(一)- 本地部署(Docker)
bisheng系列(二)- 本地部署(前后端)
二、停止前后端的docker
bisheng共计9个容器,需要独立代码启动前后端,则需要将前后端的容器停止
1、停止并删除所有容器
命令:
docker-compose down
效果
jack@Laptop-L14-gen4:~/bisheng-blog/bisheng/docker/bisheng-main/docker$ docker-compose down
Stopping bisheng-frontend ... done
Stopping bisheng-backend ... done
Stopping milvus-standalone ... done
Stopping bisheng-es ... done
Stopping bisheng-office ... done
Stopping milvus-etcd ... done
Stopping bisheng-mysql ... done
Stopping bisheng-redis ... done
Stopping milvus-minio ... done
Removing bisheng-frontend ... done
Removing bisheng-backend ... done
Removing milvus-standalone ... done
Removing bisheng-es ... done
Removing bisheng-office ... done
Removing milvus-etcd ... done
Removing bisheng-mysql ... done
Removing bisheng-redis ... done
Removing milvus-minio ... done
Removing network docker_default
jack@Laptop-L14-gen4:~/bisheng-blog/bisheng/docker/bisheng-main/docker$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5257cf9b2426 qdrant/qdrant "./entrypoint.sh" 3 weeks ago Exited (255) 2 weeks ago 0.0.0.0:6333-6334->6333-6334/tcp, :::6333-6334->6333-6334/tcp distracted_lamport
b28179c3b79b qdrant/qdrant "./entrypoint.sh" 3 weeks ago Exited (255) 3 weeks ago 0.0.0.0:6333-6334->6333-6334/tcp, :::6333-6334->6333-6334/tcp bold_beaver
b00d21f614f7 hello-world "/hello" 3 weeks ago Exited (0) 3 weeks ago objective_khayyam
114b2a816fae ubuntu1:8080 "/bin/sh -c 'npm i &…" 7 weeks ago Exited (255) 7 weeks ago 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp light_house
通过上述命令说明,bisheng的9个容器停止并删除成功
2、配置docker-compse
通过编辑docker-compose.yml文件,删除前后端的启动项,修改后内容如下:
services:mysql:container_name: bisheng-mysqlimage: mysql:8.0ports:- "3306:3306"environment:MYSQL_ROOT_PASSWORD: "1234" # 数据库密码,如果修改需要同步修改bisheng/congfig/config.yaml配置database_url的mysql连接密码MYSQL_DATABASE: bishengTZ: Asia/Shanghaivolumes:- ${DOCKER_VOLUME_DIRECTORY:-.}/mysql/conf/my.cnf:/etc/mysql/my.cnf- ${DOCKER_VOLUME_DIRECTORY:-.}/mysql/data:/var/lib/mysqlhealthcheck:test: ["CMD-SHELL", "exit | mysql -u root -p$$MYSQL_ROOT_PASSWORD"]start_period: 30sinterval: 20stimeout: 10sretries: 4restart: on-failureredis:container_name: bisheng-redisimage: redis:7.0.4ports:- "6379:6379"environment:TZ: Asia/Shanghaivolumes:- ${DOCKER_VOLUME_DIRECTORY:-.}/data/redis:/data- ${DOCKER_VOLUME_DIRECTORY:-.}/redis/redis.conf:/etc/redis.confcommand: redis-server /etc/redis.confhealthcheck:test: ["CMD-SHELL", 'redis-cli ping|grep -e "PONG\|NOAUTH"']interval: 10stimeout: 5sretries: 3restart: on-failureoffice:container_name: bisheng-officeimage: onlyoffice/documentserver:7.1.1ports:- "8701:80"environment:TZ: Asia/ShanghaiJWT_ENABLED: "false"volumes:- ${DOCKER_VOLUME_DIRECTORY:-.}/office/bisheng:/var/www/onlyoffice/documentserver/sdkjs-plugins/bishengcommand: bash -c "supervisorctl restart all"restart: on-failureelasticsearch:container_name: bisheng-esimage: docker.io/bitnami/elasticsearch:8.12.0user: rootports:- "9200:9200"- "9300:9300"environment:TZ: Asia/Shanghaivolumes:- ${DOCKER_VOLUME_DIRECTORY:-.}/data/es:/bitnami/elasticsearch/datarestart: on-failureetcd:container_name: milvus-etcdimage: quay.io/coreos/etcd:v3.5.5environment:ETCD_AUTO_COMPACTION_MODE: revisionETCD_AUTO_COMPACTION_RETENTION: "1000"ETCD_QUOTA_BACKEND_BYTES: "4294967296"ETCD_SNAPSHOT_COUNT: "50000"TZ: Asia/Shanghaivolumes:- ${DOCKER_VOLUME_DIRECTORY:-.}/data/milvus-etcd:/etcdcommand: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcdrestart: on-failurehealthcheck:test: ["CMD", "etcdctl", "endpoint", "health"]interval: 30stimeout: 20sretries: 3minio:container_name: milvus-minioimage: minio/minio:RELEASE.2023-03-20T20-16-18Zenvironment:MINIO_ACCESS_KEY: minioadminMINIO_SECRET_KEY: minioadminports:- "9100:9000"- "9101:9001"volumes:- /etc/localtime:/etc/localtime:ro- ${DOCKER_VOLUME_DIRECTORY:-.}/data/milvus-minio:/minio_datacommand: minio server /minio_data --console-address ":9001"restart: on-failurehealthcheck:test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]interval: 30stimeout: 20sretries: 3milvus:container_name: milvus-standaloneimage: milvusdb/milvus:v2.3.3command: ["milvus", "run", "standalone"]security_opt:- seccomp:unconfinedenvironment:ETCD_ENDPOINTS: etcd:2379MINIO_ADDRESS: minio:9000volumes:- /etc/localtime:/etc/localtime:ro- ${DOCKER_VOLUME_DIRECTORY:-.}/data/milvus:/var/lib/milvusrestart: on-failurehealthcheck:test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"]start_period: 90sinterval: 30stimeout: 20sretries: 3ports:- "19530:19530"- "9091:9091"depends_on:- etcd- minio
3、启动其余容器
jack@Laptop-L14-gen4:~/bisheng-blog/bisheng/docker/bisheng-main/docker$ docker-compose up -d
Creating network "docker_default" with the default driver
Creating milvus-etcd ... done
Creating bisheng-redis ... done
Creating bisheng-mysql ... done
Creating bisheng-office ... done
Creating milvus-minio ... done
Creating bisheng-es ... done
Creating milvus-standalone ... done
4、前后端代码
前后端的代码分别时:
后端:bisheng/src/backend
前端:bisheng/src/frontend
三、后端启动
通过修改部分文件,使后端可以连接到相关的容器服务
后端在open Euler 23.03启动,我使用vscode来修改相关文件
1、克隆源码
git clone https://github.com/dataelement/bisheng.git
2、安装相关环境
sudo yum -y install python3-pip python3-devel gcc g++
3、安装poetry
pip install poetry==1.8.2
4、项目环境
# 在当前项目根目录下生成环境
poetry config virtualenvs.create true# 安装环境所需的模块
poetry install# 在openEuler环境需要该包
poetry add opencv-python-headless
安装后,看下安装过程是否存在异常,如下为正常
...... - Installing types-pillow (9.5.0.6)- Installing types-pyyaml (6.0.12.20250516)- Installing types-requests (2.32.0.20250515)
......Writing lock fileInstalling the current project: bisheng (1.1.1)
5、配置文件
此处配置文件特指其余七个容器的连接配置
将/bisheng/docker/bisheng/config/config.yaml 复制到指定目录下,并修改如下:
# 修改后的文件内容,删除了注释的内容database_url:"mysql+pymysql://root:gAAAAABlp4b4c59FeVGF_OQRVf6NOUIGdxq8246EBD-b0hdK_jVKRs1x4PoAn0A6C5S6IiFKmWn0Nm5eBUWu-7jxcqw6TiVjQA==@0.0.0.0:3306/bisheng?charset=utf8mb4"# 缓存配置 redis://[[username]:[password]]@localhost:6379/0
# 普通模式:
redis_url: "redis://0.0.0.0:6379/1"# celery的broken地址
celery_redis_url: "redis://0.0.0.0:6379/2"# 知识库的milvus和es配置 支持使用 !env ${PATH} 填写环境变量的值, 若环境变量不存在则会报错
vector_stores:milvus:connection_args: '{"host":"0.0.0.0","port":"9530","user":"","password":"","secure":false}'is_partition: truepartition_suffix: "1"elasticsearch:url: 'http://0.0.0.0:9200'ssl_verify: '{"basic_auth": ("elastic", "elastic")}'# 对象存储, 目前只支持minio
object_storage:type: miniominio:schema: falsecert_check: falseendpoint: '0.0.0.0:9100'sharepoint: '0.0.0.0:9100'access_key: minioadminsecret_key: minioadminenvironment:env: devuns_support: ['png','jpg','jpeg','bmp','doc', 'docx', 'ppt', 'pptx', 'xls', 'xlsx', 'txt', 'md', 'html', 'pdf', 'csv', 'tiff']# 可根据loguru的文档配置不同 handlers
logger_conf:# 默认输出到sys.stdout的日志级别, 大于等于此级别都会输出level: DEBUG# 默认输出格式format: '<level>[{time:YYYY-MM-DD HH:mm:ss.SSSSSS}] [{level.name} process-{process.id}-{thread.id} {name}:{line}]</level> - <level>trace={extra[trace_id]} {message}</level>'# 参考loguru.add()中的参数可以配置多个handlerhandlers:# 文件路径,支持插入一些系统环境变量,若环境变量不存在则置空。例如 HOSTNAME: 主机名。后端会处理环境变量的替换- sink: "/home/jack/bisheng_11/backend/data/bisheng.log"# 日志级别level: INFO# 日志格式化函数,extra内支持trace_idformat: '<level>[{time:YYYY-MM-DD HH:mm:ss.SSSSSS}] [{level.name} process-{process.id}-{thread.id} {name}:{line}]</level> - <level>trace={extra[trace_id]} {message}</level>'# 每天的几点进行切割rotation: "00:00"retention: "3 Days"enqueue: ture- sink: "/home/jack/bisheng_11/backend/data/statistic.log"level: INFO# 和原生不一样,后端会将配置使用eval()执行转为函数用来过滤特定日志级别。推荐lambdafilter: "lambda record: record['level'].name == 'INFO' and record['message'].startswith('k=s')"format: "[{time:YYYY-MM-DD HH:mm:ss.SSSSSS}]|{level}|BISHENG|{extra[trace_id]}||{process.id}|{thread.id}|||#EX_ERR:POS={name},line {line},ERR=500,EMSG={message}"rotation: "00:00"retention: "3 Days"enqueue: ture
修改目的是将链接修改为0.0.0.0:端口,由于之前使用docker配置可以连接,但是当前由于使用代码直接启动,无法识别http://mysql:3306这种链接,故而需要修改
日志路径修改是为了保证权限的问题
编辑bisheng/src/backend/bisheng/api/services/assistant_agent.py,修改文件路径
# 原内容async def record_chat_history(self, message: List[Any]):# 记录助手的聊天历史if not os.getenv('BISHENG_RECORD_HISTORY'):returntry:os.makedirs('/app/data/history', exist_ok=True)with open(f'/app/data/history/{self.assistant.id}_{time.time()}.json',# 修改后async def record_chat_history(self, message: List[Any]):# 记录助手的聊天历史if not os.getenv('BISHENG_RECORD_HISTORY'):returntry:os.makedirs('/home/jack/bisheng/src/backend/data/history', exist_ok=True)with open(f'/home/jack/bisheng/src/backend/data/history/{self.assistant.id}_{time.time()}.json',
处理fastapi_jwt_auth、langchain_openai的兼容性
patch -p1 < ./bisheng/patches/fastapi_jwt_auth.patch ./.venv/lib/python3.10/site-packages/fastapi_jwt_auth/config.pypatch -p1 < ./bisheng/patches/langchain_openai.patch ./.venv/lib/python3.10/site-packages/langchain_openai/chat_models/base.py
6、启动项目
uvicorn bisheng.main:app --host 0.0.0.0 --port 7860
成功如下:
.
.
.
INFO: Started server process [64315]
INFO: Waiting for application startup.
[2025-05-19 11:25:20.152105] [2025-05-19 11:25:20.151985] [DEBUG process-64315-139958058770496 bisheng.interface.utils:91] - trace=1 Setting up LLM caching with InMemoryCache
[2025-05-19 11:25:20.153851] [2025-05-19 11:25:20.153779] [INFO process-64315-139958058770496 bisheng.interface.utils:93] - trace=1 LLM caching setup with InMemoryCache
[2025-05-19 11:25:20.189322] [2025-05-19 11:25:20.189129] [DEBUG process-64315-139958058770496 bisheng.database.service:59] - trace=1 Creating database and tables
[2025-05-19 11:25:20.564080] [2025-05-19 11:25:20.563913] [DEBUG process-64315-139958058770496 bisheng.database.service:79] - trace=1 Database and tables created successfully
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:7860 (Press CTRL+C to quit)
四、前端启动
略(参考上一节中的前端配置即可)