当前位置： 首页 > news >正文

Dify本地 + vllm + Qwen

news 2025/7/4 3:45:22

注意：v100显卡会出现不适配，不推荐使用

 1. 安装 Docker

ubuntu 22.04 docker 安装&使用_ubuntu22.04 安装docker-CSDN博客

 2. 安装vllm

pip install -U xformers torch torchvision torchaudio triton --index-url https://download.pytorch.org/whl/cu121
pip install modelscope vllm

3. 下载Qwen开源模型

#模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('Qwen/Qwen2.5-0.5B-Instruct')

4. 部署Dify到本地

git clone https://github.com/langgenius/dify

启动 Dify

  1. 进入 Dify 源代码的 Docker 目录

    cd dify/docker

  2. 复制环境配置文件

    cp .env.example .env

  3. 启动 Docker 容器

    根据你系统上的 Docker Compose 版本，选择合适的命令来启动容器。你可以通过 $ docker compose version 命令检查版本，详细说明请参考 Docker 官方文档：

    • 如果版本是 Docker Compose V2，使用以下命令：
    docker compose up -d
    • 如果版本是 Docker Compose V1，使用以下命令：
    docker-compose up -d

运行命令后，你应该会看到类似以下的输出，显示所有容器的状态和端口映射，通过这些步骤，你可以在本地成功安装 Dify。

修改端口：

更新 Dify

进入 dify 源代码的 docker 目录，按顺序执行以下命令：

cd dify/docker
docker compose down
git pull origin main
docker compose pull
docker compose up -d

访问 Dify

你可以先前往管理员初始化页面设置设置管理员账户：

# 本地环境
http://localhost/install# 服务器环境
http://your_server_ip/install

Dify 主页面：

# 本地环境
http://localhost# 服务器环境
http://your_server_ip

 5. 启动 vLLM 的 OpenAI 兼容服务

如果使用云服务器请先确认暴露端口

以闪电云为例：

sudo apt install rinetd
echo "0.0.0.0 8880 127.0.0.1 9000" > /etc/rinetd.conf
sudo systemctl restart rinetd

启动vllm服务 

命令行输入 vllm serve Qwen/Qwen2.5-1.5B-Instruct --port 9999 --dtype float16

# 使用绝对路径
vllm serve /root/.cache/modelscope/hub/models/Qwen/Qwen2.5-0.5B-Instruct --port 8000 --dtype float16

连通性测试

使用ifconfig显示每个网卡的 IP 地址、MAC 地址、子网掩码、广播地址

root@9gpu-com:~/dify/docker# ifconfig
br-2796d2728d26: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet 172.18.0.1  netmask 255.255.0.0  broadcast 172.18.255.255inet6 fe80::d8b4:7bff:fe93:2a82  prefixlen 64  scopeid 0x20<link>ether da:b4:7b:93:2a:82  txqueuelen 0  (Ethernet)RX packets 5628  bytes 5128258 (5.1 MB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 5625  bytes 8112383 (8.1 MB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0br-c38714503e51: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet 172.19.0.1  netmask 255.255.0.0  broadcast 172.19.255.255inet6 fe80::48f5:b4ff:fec6:7c38  prefixlen 64  scopeid 0x20<link>ether 4a:f5:b4:c6:7c:38  txqueuelen 0  (Ethernet)RX packets 13  bytes 364 (364.0 B)RX errors 0  dropped 0  overruns 0  frame 0TX packets 50  bytes 6315 (6.3 KB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255ether 66:42:9b:df:48:d2  txqueuelen 0  (Ethernet)RX packets 0  bytes 0 (0.0 B)RX errors 0  dropped 0  overruns 0  frame 0TX packets 0  bytes 0 (0.0 B)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0enp1s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet 192.168.122.200  netmask 255.255.255.0  broadcast 192.168.122.255inet6 fe80::5054:ff:fe4a:4c79  prefixlen 64  scopeid 0x20<link>ether 52:54:00:4a:4c:79  txqueuelen 1000  (Ethernet)RX packets 1116491  bytes 2054627970 (2.0 GB)RX errors 0  dropped 9  overruns 0  frame 0TX packets 570243  bytes 54255571 (54.2 MB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536inet 127.0.0.1  netmask 255.0.0.0inet6 ::1  prefixlen 128  scopeid 0x10<host>loop  txqueuelen 1000  (Local Loopback)RX packets 73435  bytes 14470967 (14.4 MB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 73435  bytes 14470967 (14.4 MB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0veth0033170: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet6 fe80::ecba:63ff:fe64:74cf  prefixlen 64  scopeid 0x20<link>ether ee:ba:63:64:74:cf  txqueuelen 0  (Ethernet)RX packets 4591  bytes 5453398 (5.4 MB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 3742  bytes 5384255 (5.3 MB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0veth1592100: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet6 fe80::e83a:acff:fe54:fae6  prefixlen 64  scopeid 0x20<link>ether ea:3a:ac:54:fa:e6  txqueuelen 0  (Ethernet)RX packets 2829  bytes 1585191 (1.5 MB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 2575  bytes 1796899 (1.7 MB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0veth128db92: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet6 fe80::9012:8bff:fead:cf8  prefixlen 64  scopeid 0x20<link>ether 92:12:8b:ad:0c:f8  txqueuelen 0  (Ethernet)RX packets 3  bytes 126 (126.0 B)RX errors 0  dropped 0  overruns 0  frame 0TX packets 76  bytes 8993 (8.9 KB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0veth397ec04: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet6 fe80::bc8e:99ff:fe39:fdec  prefixlen 64  scopeid 0x20<link>ether be:8e:99:39:fd:ec  txqueuelen 0  (Ethernet)RX packets 1210  bytes 4802462 (4.8 MB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 1536  bytes 264886 (264.8 KB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0veth3d1bfb2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet6 fe80::8cc8:1fff:fe8c:6fb3  prefixlen 64  scopeid 0x20<link>ether 8e:c8:1f:8c:6f:b3  txqueuelen 0  (Ethernet)RX packets 3  bytes 126 (126.0 B)RX errors 0  dropped 0  overruns 0  frame 0TX packets 77  bytes 9035 (9.0 KB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0veth5ab3c2e: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet6 fe80::b0dd:f4ff:fea0:45cd  prefixlen 64  scopeid 0x20<link>ether b2:dd:f4:a0:45:cd  txqueuelen 0  (Ethernet)RX packets 17  bytes 1097 (1.0 KB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 107  bytes 11337 (11.3 KB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0veth838cf02: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet6 fe80::784e:60ff:feba:be8c  prefixlen 64  scopeid 0x20<link>ether 7a:4e:60:ba:be:8c  txqueuelen 0  (Ethernet)RX packets 1546  bytes 726654 (726.6 KB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 2128  bytes 762270 (762.2 KB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0veth8555bb5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet6 fe80::340e:6fff:fe90:eccc  prefixlen 64  scopeid 0x20<link>ether 36:0e:6f:90:ec:cc  txqueuelen 0  (Ethernet)RX packets 284  bytes 755628 (755.6 KB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 567  bytes 775220 (775.2 KB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0veth94a620c: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet6 fe80::14a2:97ff:fe89:be32  prefixlen 64  scopeid 0x20<link>ether 16:a2:97:89:be:32  txqueuelen 0  (Ethernet)RX packets 3  bytes 126 (126.0 B)RX errors 0  dropped 0  overruns 0  frame 0TX packets 120  bytes 11332 (11.3 KB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0veth9dbece6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet6 fe80::18b5:63ff:fe2a:87ea  prefixlen 64  scopeid 0x20<link>ether 1a:b5:63:2a:87:ea  txqueuelen 0  (Ethernet)RX packets 21  bytes 1412 (1.4 KB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 103  bytes 11022 (11.0 KB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0vethaa36645: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet6 fe80::74ab:f6ff:fe54:b443  prefixlen 64  scopeid 0x20<link>ether 76:ab:f6:54:b4:43  txqueuelen 0  (Ethernet)RX packets 9410  bytes 1180211 (1.1 MB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 7585  bytes 8024737 (8.0 MB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0vethaa203b0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet6 fe80::284b:9fff:fe41:44f6  prefixlen 64  scopeid 0x20<link>ether 2a:4b:9f:41:44:f6  txqueuelen 0  (Ethernet)RX packets 2963  bytes 951071 (951.0 KB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 3604  bytes 746609 (746.6 KB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0vethad67544: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet6 fe80::419:99ff:fe34:d7d  prefixlen 64  scopeid 0x20<link>ether 06:19:99:34:0d:7d  txqueuelen 0  (Ethernet)RX packets 6937  bytes 1036251 (1.0 MB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 8841  bytes 1719138 (1.7 MB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
1. br-2796d2728d26、br-c38714503e51、docker0
  • 这些是 Docker 创建的虚拟网桥（bridge），用于容器之间的网络通信。
  • 例如：
  • br-2796d2728d26 的 IP：172.18.0.1
  • br-c38714503e51 的 IP：172.19.0.1
  • docker0 的 IP：172.17.0.1
  • 这些 IP 只在本机和 Docker 容器之间通信时有用，不是你主机的内网或公网 IP。
2. enp1s0
  • 这是你主机的物理网卡（有线网卡）。
  • inet 192.168.122.200 这是你主机在局域网（内网）中的 IP 地址。
  • 你在同一局域网的其他设备可以通过 192.168.122.200 访问你的主机（前提是防火墙和端口已放行）。

 

6.使用OpenAl-API-compatible插件在内网部署

http://www.xdnf.cn/news/289657.html

相关文章：

  • 走出 Demo，走向现实：DeepSeek-VL 的多模态工程路线图
  • 协议(消息)生成
  • qt csv文件写操作
  • 3.5/Q1，GBD数据库最新一区文章解读
  • P3246 [HNOI2016] 序列 Solution
  • 项目生成日志链路id，traceId
  • 数据库的二级索引
  • 【PostgreSQL数据分析实战：从数据清洗到可视化全流程】4.4 异构数据源整合（CSV/JSON/Excel数据导入）
  • 数据结构小扫尾——栈
  • BUUCTF 大流量分析（一） 1
  • 比较两种判断相同二叉树的方法：递归与遍历序列对比
  • 湖北理元理律师事务所：债务法律服务的民生价值重构
  • brep2seq kaggle安装 micromamba conda环境
  • 政务服务智能化改造方案和案例分析
  • QT6 源（79）：阅读与注释表盘 QDial 的源码，其是基类QAbstractSlider 的子类，及其刻度线的属性举例
  • 深入理解 CSS Flex 布局：代码实例解析
  • 【Hive入门】Hive安全管理与权限控制：基于SQL标准的授权GRANT REVOKE深度解析
  • kotlin 05flow -从 LiveData 迁移到 Kotlin Flow 完整教程
  • 用Python打造自己的专属命令行工具
  • 手写 Vue 源码 === reactive 方法
  • 判断题材持续性
  • Redis的过期设置和策略
  • rocketMq win10安装并注册服务 centos7安装
  • 嵌软笔记名词汇总
  • C++基础知识1：内建数据类型 vs 自定义数据类型
  • 豆包多轮对话优化策略：上下文理解与记忆,意图识别，对话管理
  • 力扣每日一题1128等价多米诺骨牌对的数量
  • C++和Lua混和调用
  • 什么是加油站专用可燃气体磁吸无线检测探测器
  • 小米刷新率 2.4 | 突破屏幕刷新率限制，享受更流畅视觉体验的应用程序