Centos7基于vllm安装deepseek: nvidia显卡驱动升级，CUDA安装，Pytorch安装，vllm安装

SE_You

2025-3-6

CentOS

最近准备deepseek比较火，准备本地搭建一下。

注意：需要vpn，vpn的流量要足够大，足够下载模型文件及相关的软件。如果没有可以看评论区推荐。

一、环境查看
提供了一台独立的机器配置很高，说是还有一块80G的A100显卡。这台机器之前有人在上面已经安装过机器学习、深度学习的软件了，但是可能有问题。

搭建deepseek环境前，一定不要着急马上安装，要先搞清楚每个软件都使用什么版本后，再开始安装，防止后面要安装的软件前面按照的软件版本不支持。

安装前，要先查看机器：

1.1.操作系统
是：CentOS 7.9.2009 x86_64

# 查看操作系统版本： CentOS 7.9.2009
(base) root@ydpslave141:/home/81097864/gup-driver# 
(base) root@ydpslave141:/home/81097864/gup-driver# lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 7.9.2009 (Core)
Release:        7.9.2009
Codename:       Core
(base) root@ydpslave141:/home/81097864/gup-driver# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
(base) root@ydpslave141:/home/81097864/gup-driver# uname -a
Linux ydpslave141.yonghui.cn 3.10.0-1160.95.1.el7.x86_64 #1 SMP Mon Jul 24 13:59:37 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
(base) root@ydpslave141:/home/81097864/gup-driver# 
(base) root@ydpslave141:/home/81097864/gup-driver#

1.2.GPU硬件

说是A100，但是看不到具体是什么型号，只能看到参数NVIDIA Corporation Device 20b5 (rev a1)

# 查看是否已经安装GPU  及GPU的型号    及是否安装GPU驱动    已经CUDA
(base) root@ydpslave141:/home/81097864/gup-driver# 
(base) root@ydpslave141:/home/81097864/gup-driver# lspci | grep -i vga
02:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200e [Pilot] ServerEngines (SEP1) (rev 42)
(base) root@ydpslave141:/home/81097864/gup-driver# 
# 查看硬件显卡的型号   20b5就是A100显卡
(base) root@ydpslave141:/home/81097864/gup-driver# lspci | grep -i nvidia
65:00.0 3D controller: NVIDIA Corporation Device 20b5 (rev a1)
(base) root@ydpslave141:/home/81097864/gup-driver#

这就要到官方去查看了，登陆网站，输入上面的参数：20b5，可以看到确实是A100。

https://admin.pci-ids.ucw.cz/mods/PC/10de?action=help?help=pci

1.3.GPU驱动

执行nvidia-smi命令，如果不显示，说明没有安装驱动，如果显示了，说明已经安装了驱动。

这里可以看出GPU已经安装的驱动版本是465.19.01，当前能支持的最高cuda版本是11.3

(base) root@ydpslave141:/home/81097864/gup-driver# 
# 查看nvidiaGPU已经安装的驱动版本是465.19.01，当前能支持的最高cuda版本是11.3
# 如果没有安装驱动 命令执行失败
(base) root@ydpslave141:/home/81097864/gup-driver# nvidia-smi
Fri Feb 14 15:58:21 2025       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.19.01    Driver Version: 465.19.01    CUDA Version: 11.3     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA Graphics...  Off  | 00000000:65:00.0 Off |                    0 |
| N/A   39C    P0    43W / 300W |      3MiB / 81251MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
(base) root@ydpslave141:/home/81097864/gup-driver# 
(base) root@ydpslave141:/home/81097864/gup-driver# 
(base) root@ydpslave141:/home/81097864/gup-driver# 
(base) root@ydpslave141:/home/81097864/gup-driver# nvidia-smi -a | head -n 30

==============NVSMI LOG==============

Timestamp                                 : Fri Feb 14 15:58:36 2025
Driver Version                            : 465.19.01
CUDA Version                              : 11.3

Attached GPUs                             : 1
GPU 00000000:65:00.0
    Product Name                          : NVIDIA Graphics Device
    Product Brand                         : NVIDIA
    Display Mode                          : Disabled
    Display Active                        : Disabled
    Persistence Mode                      : Disabled
    MIG Mode
        Current                           : Disabled
        Pending                           : Disabled
    Accounting Mode                       : Disabled
    Accounting Mode Buffer Size           : 4000
    Driver Model
        Current                           : N/A
        Pending                           : N/A
    Serial Number                         : 1324321032825
    GPU UUID                              : GPU-88948629-c391-6f5f-00e9-950a77371723
    Minor Number                          : 0
    VBIOS Version                         : 92.00.68.00.01
    MultiGPU Board                        : No
    Board ID                              : 0x6500
    GPU Part Number                       : 900-21001-2720-030
    Inforom Version
(base) root@ydpslave141:/home/81097864/gup-driver# 
(base) root@ydpslave141:/home/81097864/gup-driver#

1.4.CUDA软件

# 查询已经安装的cuda的版本号
(base) root@ydpslave141:/home/81097864/gup-driver# nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
(base) root@ydpslave141:/home/81097864/gup-driver#

这里可以看到cuda的版本是11.8，说明之前的环境就是有问题的，之前的驱动能支持的最大cuda版本是11.3，现在实际安装的是11.8。

1.5.pytorch和vllm
这两个软件就不用看了，因为后续要新建conda虚拟环境，要重新安装。

============================================

二、安装规划
一定要规划好再安装。不要走一步看一步。因为可能前面几个软件安装完成了，找不到可用的下一个软件的可用版本了。

软件	版本	备注
GPU驱动	550.144.03	支持的cuda最大12.4
CUDA	12.4	centos系统的cuda最高只有12.4
pytorch	官网页面选择
vllm	官网页面选择

============================================

三、环境安装
因为已经有驱动了，现有的驱动能支持的最高CUDA版本是11.3，不满足deepseek的要求，所以要升级。要先卸载驱动再升级驱动。

卸载是个问题，卸载驱动和CUDA前，要先知道CUDA和驱动是通过什么方式安装的。

安装有多种方式yum、rpm、以及官方的.run文件，不同的安装方式有不同的卸载方式。

3.1.卸载CUDA
在 CentOS 7 上卸载 CUDA 前，需先确认 CUDA 的安装方式（yum、rpm 或 .run 文件）。以下是不同安装方式的详细卸载步骤：

1. 确认 CUDA 安装方式
(1) 检查是否通过 yum 或 rpm 安装

rpm -qa | grep -i cuda

如果输出类似 cuda-11-7 或 cuda-drivers-xxx 的包名，则为 yum 或 rpm 安装。
若无输出，可能是通过 .run 文件安装。
(2) 检查是否通过 .run 文件安装

ls /usr/local | grep cuda

若存在 /usr/local/cuda-<版本号> 目录且无相关 rpm 包，则为 .run 文件安装。
2. 根据安装方式卸载 CUDA
方式一：通过 yum 安装的卸载

列出所有 CUDA 相关包：

yum list installed | grep cuda

卸载所有相关包（以 cuda-11-7 为例）

sudo yum remove cuda-11-7

清理残留依赖：

sudo yum autoremove

方式二：通过 rpm 包安装的卸载

查找已安装的 CUDA 包：

rpm -qa | grep -i cuda

卸载所有相关包（以 cuda-11-7 为例）：

sudo rpm -e cuda-11-7

手动删除残留文件和配置：

sudo rm -rf /usr/local/cuda*

方式三：通过 .run 文件安装的卸载

运行 CUDA 安装脚本的卸载功能

sudo /usr/local/cuda-<版本号>/bin/cuda-uninstaller

根据提示选择卸载选项（通常需要选择全部组件）。
手动删除残留文件：

sudo rm -rf /usr/local/cuda-<版本号>

清理环境变量（编辑 ~/.bashrc 或 /etc/profile）

sudo sed -i '/cuda/d' ~/.bashrc /etc/profile
source ~/.bashrc

3.2.卸载GPU驱动
以下是卸载 NVIDIA GPU 驱动的详细步骤，根据不同安装方式分类说明：

1.确认驱动安装方式
在卸载前，先通过以下方法确认驱动的安装来源：

#检查是否通过 yum 安装
yum list installed | grep nvidia

# 检查是否通过 rpm 安装
rpm -qa | grep -i nvidia

# 检查是否通过 .run 文件安装（查找安装日志）
这种方式通常会创建一个日志文件，可以通过查看 /var/log/nvidia-installer.log 来确认是否使用了 .run 文件进行安装

2. 不同安装方式的卸载
方法一：通过 yum安装

# 查看已安装的 NVIDIA 相关包
yum list installed | grep -i nvidia 

# 卸载所有 NVIDIA 驱动包
sudo yum remove nvidia-driver* *nvidia*  

# 清理残留依赖和缓存
sudo yum autoremove && sudo yum clean all

方法二：通过 rpm 包手动安装

适用于直接下载 .rpm 文件安装的情况。

# 列出所有 NVIDIA 相关的 rpm 包
rpm -qa | grep -i nvidia

# 卸载指定的包（替换为上一步中显示的实际包名）
sudo rpm -e nvidia-driver-xxx.x86_64

# 如果存在依赖问题，强制卸载（慎用）
sudo rpm -e --nodeps nvidia-driver-xxx.x86_64

方法三：通过官方 .run 文件安装

# 进入 NVIDIA 驱动安装文件所在目录
cd ~/Downloads  # 假设驱动文件在 Downloads 目录

# 执行卸载命令（需要原 .run 文件）
sudo ./NVIDIA-Linux-x86_64-xxx.xx.run --uninstall

# 如果原 .run 文件已丢失，重新下载相同版本后卸载

或直接运行内置卸载脚本（如有）：

sudo /usr/bin/nvidia-uninstall

3.通用清理步骤
卸载后均需执行以下操作：

删除残留文件和配置

sudo rm -rf /etc/X11/xorg.conf.d/00-nvidia.conf
sudo rm -rf /usr/lib/xorg/modules/drivers/nvidia_drv*
sudo rm -rf /usr/lib/modules/*/kernel/drivers/video/nvidia

禁用 Nouveau 驱动（如果之前启用）

# 删除或重命名黑名单配置文件
sudo rm /etc/modprobe.d/blacklist-nouveau.conf

# 重新生成 initramfs
sudo dracut --force

4. 重启系统

sudo reboot

5. 验证卸载结果
重启后检查驱动是否已卸载：

# 检查 NVIDIA 内核模块
lsmod | grep -i nvidia

# 检查 GPU 信息（应显示为开源驱动或无信息）
nvidia-smi  # 若提示命令未找到，则卸载成功

按以上步骤操作后，NVIDIA 驱动将被完全卸载。

3.3.GPU驱动
登陆官网，根据驱动型号选择对应的驱动下载。

https://www.nvidia.cn/drivers/lookup/

下载

(base) root@ydpslave141:/home/81097864/gup-driver# wget https://cn.download.nvidia.com/tesla/550.144.03/NVIDIA-Linux-x86_64-550.144.03.run
--2025-02-14 16:10:00--  https://cn.download.nvidia.com/tesla/550.144.03/NVIDIA-Linux-x86_64-550.144.03.run
Connecting to 10.67.58.139:1080... connected.
Proxy request sent, awaiting response... 200 OK
Length: 307272669 (293M) [application/octet-stream]
Saving to: ‘NVIDIA-Linux-x86_64-550.144.03.run’

100%[=================================================================================================================================>] 307,272,669 2.19MB/s   in 2m 18s 

2025-02-14 16:12:20 (2.13 MB/s) - ‘NVIDIA-Linux-x86_64-550.144.03.run’ saved [307272669/307272669]

(base) root@ydpslave141:/home/81097864/gup-driver#

下载后是个NVIDIA-Linux-x86_64-550.144.03.run，这个是个run的可执行文件。

安装

(base) root@ydpslave141:/home/81097864/gup-driver# 
# 添加可执行权限
(base) root@ydpslave141:/home/81097864/gup-driver# chmod 777 ./NVIDIA-Linux-x86_64-550.144.03.run
(base) root@ydpslave141:/home/81097864/gup-driver# 
# 执行安装：
(base) root@ydpslave141:/home/81097864/gup-driver# ./NVIDIA-Linux-x86_64-550.144.03.run

如果之前卸载驱动失败，安装时报错：

正常安装结束：

验证

root@ydpslave141:/home/81097864/gup-driver# 
root@ydpslave141:/home/81097864/gup-driver# nvidia-smi
Fri Feb 14 23:38:23 2025       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.144.03             Driver Version: 550.144.03     CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA A100 80GB PCIe          Off |   00000000:65:00.0 Off |                    0 |
| N/A   42C    P0             64W /  300W |       1MiB /  81920MiB |      3%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
root@ydpslave141:/home/81097864/gup-driver#

3.4.CUDA安装
Centos版本可以下载的CUDA最高版本就是12.4

驱动和CUDA的版本对应关系：https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/

CUDA下载：https://developer.nvidia.com/cuda-toolkit-archive

我们选择run file格式下载

下载

root@ydpslave141:/home/81097864/cuda# 
root@ydpslave141:/home/81097864/cuda# wget https://developer.download.nvidia.com/compute/cuda/12.4.0/local_installers/cuda_12.4.0_550.54.14_linux.run
--2025-02-14 17:59:17--  https://developer.download.nvidia.com/compute/cuda/12.4.0/local_installers/cuda_12.4.0_550.54.14_linux.run
Connecting to 10.67.58.139:1080... connected.
Proxy request sent, awaiting response... 200 OK
Length: 4454353277 (4.1G) [application/octet-stream]
Saving to: ‘cuda_12.4.0_550.54.14_linux.run’

99% [==============================================================================================================================> ] 4,454,350,848 2.78MB/s   in 26m 4s 

2025-02-14 18:25:23 (2.72 MB/s) - Connection closed at byte 4454350848. Retrying.

--2025-02-14 18:25:24--  (try: 2)  https://developer.download.nvidia.com/compute/cuda/12.4.0/local_installers/cuda_12.4.0_550.54.14_linux.run
Connecting to 10.67.58.139:1080... connected.
Proxy request sent, awaiting response... 206 Partial Content
Length: 4454353277 (4.1G), 2429 (2.4K) remaining [application/octet-stream]
Saving to: ‘cuda_12.4.0_550.54.14_linux.run’

100%[+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>] 4,454,353,277 --.-K/s   in 0s      

2025-02-14 18:25:25 (340 MB/s) - ‘cuda_12.4.0_550.54.14_linux.run’ saved [4454353277/4454353277]

root@ydpslave141:/home/81097864/cuda#

安装

下载后也是run文件cuda_12.4.0_550.54.14_linux.run，赋上可执行权限后安装。

安装过程中有几个需要和用户交换的地方：选择安装组件，不要选driver，因为驱动已经安装。

root@ydpslave141:/home/81097864/cuda# 
# 赋上可执行权限
root@ydpslave141:/home/81097864/cuda# chmod 777 cuda_12.4.0_550.54.14_linux.run 
# 安装
root@ydpslave141:/home/81097864/cuda# ./cuda_12.4.0_550.54.14_linux.run 
m
0m End User License Agreement                                                  0m
0m --------------------------                                                  0m
0m                                                                             0m
0m NVIDIA Software License Agreement and CUDA Supplement to                    0m
0m Software License Agreement.                                                 0m
0m                                                                             0m
0m The CUDA Toolkit End User License Agreement applies to the                  0m
0m NVIDIA CUDA Toolkit, the NVIDIA CUDA Samples, the NVIDIA                    0m
0m Display Driver, NVIDIA Nsight tools (Visual Studio Edition),                0m
0m and the associated documentation on CUDA APIs, programming                  0m
0m model and development tools. If you do not agree with the                   0m
0m terms and conditions of the license agreement, then do not                  0m
0m download or use the software.                                               0m
0m                                                                             0m
0m Last updated: January 12, 2024.                                             0m
0m                                                                             0m
0m                                                                             0m
0m Preface                                                                     0m
0m -------                                                                     0m
0m                                                                             0m
0m0m                                                                           0m
0mDo you accept the above EULA? (accept/decline/quit):                         0m
0maccept      #这里需要输入accept                                                0m
m
m
0mCUDA Installer             t                                                 0m
0m- [ ] Driver               -                                                 0m
0m     [ ] 550.54.14                                                           0m
0m+ [X] CUDA Toolkit 12.4                                  o                   0m
0m  [X] CUDA Demo Suite 12.4  .                                                0m
0m  [X] CUDA Documentation 12.4                                                0m
0m- [ ] Kernel Objects                                       e                 0m
0m     [ ] nvidia-fs                                       A                   0m
0m  Options                                                    ,               0m
0m  Install                                                  g                 0m
0m                                                          e                  0m
0m                                                           t                 0m
0m                              .                                              0m
0m                                                                             0m
0m                                .                                            0m
0m                                                                             0m
0m                                                                             0m
0m        e                                                                    0m
0m        -                                                                    0m
0m                                                                             0m
m
0                                                     :                        0m
0mUp/Down: Move | Left/Right: Expand | 'Enter': Select | 'A': Advanced options 0m
m # 这里需要通过键盘上下键和输入键 去掉Driver  点击Install
m
0mExisting installation of CUDA Toolkit 12.4 found:                            0m
0mUpgrade all r                                                                0m
0mChoose components to upgrade                                                 0m
0mNo, abort installation 4                                                     0m
0m                          4                                                  0m
0m                             4                                               0m
0m                    s                                                        0m
0m                                                                             0m
0m         s                                                                   0m
0m                                                                             0m
0m                                                                             0m
0m                                                                             0m
0m                                                                             0m
0m                                                                             0m
0m                                                                             0m
0m                                                                             0m
0m                                                                             0m
0m                                                                             0m
0m                                                                             0m
0m                                                                             0m
0m                                                                             0m
0m                                                                             0m
0m Up/Down: Move |'Enter': Select                                             s 0m
m
===========
= Summary =
===========

Driver:   Not Selected
Toolkit:  Installed in /usr/local/cuda-12.4/

Please make sure that
 -   PATH includes /usr/local/cuda-12.4/bin
 -   LD_LIBRARY_PATH includes /usr/local/cuda-12.4/lib64, or, add /usr/local/cuda-12.4/lib64 to /etc/ld.so.conf and run ldconfig as root

To uninstall the CUDA Toolkit, run cuda-uninstaller in /usr/local/cuda-12.4/bin
***WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 550.00 is required for CUDA 12.4 functionality to work.
To install the driver using this installer, run the following command, replacing <CudaInstaller> with the name of this run file:
    sudo <CudaInstaller>.run --silent --driver

Logfile is /var/log/cuda-installer.log
root@ydpslave141:/home/81097864/cuda#

配置环境变量

#配置环境变量
(base) root@ydpslave141:/home/81097864/cuda# vim /etc/profile          
# /etc/profile
........
文件结尾加上cuda的环境变量配置
export PATH=/usr/local/cuda-12.4/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.4/lib64:$LD_LIBRARY_PATH

(base) root@ydpslave141:/home/81097864/cuda# 
root@ydpslave141:/home/81097864/cuda# source /etc/profile

验证

(deepseek_py39) root@ydpslave141:/home/81097864/cuda# nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Tue_Feb_27_16:19:38_PST_2024
Cuda compilation tools, release 12.4, V12.4.99
Build cuda_12.4.r12.4/compiler.33961263_0
(deepseek_py39) root@ydpslave141:/home/81097864/cuda#

3.4.Miniconda安装

略

3.5.Pytorch安装

进入官网：https://pytorch.org/

选择环境等信息后，获得安装命令：

https://www.imgdata.cn/imgs/2025/03/30/25db17740837bc07.png

(vllm_env) root@ydpslave141:/home/81097864/cuda# conda create -n deepseek_env python=3.10
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /opt/cloudera/parcels/Anaconda/envs/deepseek_env

  added / updated specs:
    - python=3.10


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    xz-5.6.4                   |       h5eee18b_1         567 KB  defaults
    ------------------------------------------------------------
                                           Total:         567 KB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      anaconda/pkgs/main/linux-64::_libgcc_mutex-0.1-main
  _openmp_mutex      anaconda/pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu
  bzip2              anaconda/pkgs/main/linux-64::bzip2-1.0.8-h5eee18b_6
  ca-certificates    anaconda/pkgs/main/linux-64::ca-certificates-2024.12.31-h06a4308_0
  ld_impl_linux-64   anaconda/pkgs/main/linux-64::ld_impl_linux-64-2.40-h12ee557_0
  libffi             anaconda/pkgs/main/linux-64::libffi-3.4.4-h6a678d5_1
  libgcc-ng          anaconda/pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1
  libgomp            anaconda/pkgs/main/linux-64::libgomp-11.2.0-h1234567_1
  libstdcxx-ng       anaconda/pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1
  libuuid            anaconda/pkgs/main/linux-64::libuuid-1.41.5-h5eee18b_0
  ncurses            anaconda/pkgs/main/linux-64::ncurses-6.4-h6a678d5_0
  openssl            anaconda/pkgs/main/linux-64::openssl-3.0.15-h5eee18b_0
  pip                anaconda/pkgs/main/linux-64::pip-25.0-py310h06a4308_0
  python             anaconda/pkgs/main/linux-64::python-3.10.16-he870216_1
  readline           anaconda/pkgs/main/linux-64::readline-8.2-h5eee18b_0
  setuptools         anaconda/pkgs/main/linux-64::setuptools-75.8.0-py310h06a4308_0
  sqlite             anaconda/pkgs/main/linux-64::sqlite-3.45.3-h5eee18b_0
  tk                 anaconda/pkgs/main/linux-64::tk-8.6.14-h39e8969_0
  tzdata             anaconda/pkgs/main/noarch::tzdata-2025a-h04d1e81_0
  wheel              anaconda/pkgs/main/linux-64::wheel-0.45.1-py310h06a4308_0
  xz                 anaconda/pkgs/main/linux-64::xz-5.6.4-h5eee18b_1
  zlib               anaconda/pkgs/main/linux-64::zlib-1.2.13-h5eee18b_1


Proceed ([y]/n)? y


Downloading and Extracting Packages
xz-5.6.4             | 567 KB    | ################################################################################################################################ | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate deepseek_env
#
# To deactivate an active environment, use
#
#     $ conda deactivate

(vllm_env) root@ydpslave141:/home/81097864/cuda# conda activate deepseek_env
(deepseek_env) root@ydpslave141:~# pip3 install torch torchvision torchaudio
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Collecting torch
  Using cached https://mirrors.aliyun.com/pypi/packages/37/81/aa9ab58ec10264c1abe62c8b73f5086c3c558885d6beecebf699f0dbeaeb/torch-2.6.0-cp310-cp310-manylinux1_x86_64.whl (766.7 MB)
Collecting torchvision
  Using cached https://mirrors.aliyun.com/pypi/packages/5e/44/32e2d2d174391374d5ff3c4691b802e8efda9ae27ab9062eca2255b006af/torchvision-0.21.0-cp310-cp310-manylinux1_x86_64.whl (7.2 MB)
Collecting torchaudio
  Using cached https://mirrors.aliyun.com/pypi/packages/ad/28/4dbe7e70966e16ebb90d5c887c12e3fc6d08a1c1ce0a79f8de357f0c36f9/torchaudio-2.6.0-cp310-cp310-manylinux1_x86_64.whl (3.4 MB)
Collecting filelock (from torch)
  Using cached https://mirrors.aliyun.com/pypi/packages/89/ec/00d68c4ddfedfe64159999e5f8a98fb8442729a63e2077eb9dcd89623d27/filelock-3.17.0-py3-none-any.whl (16 kB)
Collecting typing-extensions>=4.10.0 (from torch)
  Using cached https://mirrors.aliyun.com/pypi/packages/26/9f/ad63fc0248c5379346306f8668cda6e2e2e9c95e01216d2b8ffd9ff037d0/typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Collecting networkx (from torch)
  Using cached https://mirrors.aliyun.com/pypi/packages/b9/54/dd730b32ea14ea797530a4479b2ed46a6fb250f682a9cfb997e968bf0261/networkx-3.4.2-py3-none-any.whl (1.7 MB)
Collecting jinja2 (from torch)
  Using cached https://mirrors.aliyun.com/pypi/packages/bd/0f/2ba5fbcd631e3e88689309dbe978c5769e883e4b84ebfe7da30b43275c5a/jinja2-3.1.5-py3-none-any.whl (134 kB)
Collecting fsspec (from torch)
  Using cached https://mirrors.aliyun.com/pypi/packages/e2/94/758680531a00d06e471ef649e4ec2ed6bf185356a7f9fbfbb7368a40bd49/fsspec-2025.2.0-py3-none-any.whl (184 kB)
Collecting nvidia-cuda-nvrtc-cu12==12.4.127 (from torch)
  Using cached https://mirrors.aliyun.com/pypi/packages/2c/14/91ae57cd4db3f9ef7aa99f4019cfa8d54cb4caa7e00975df6467e9725a9f/nvidia_cuda_nvrtc_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (24.6 MB)
Collecting nvidia-cuda-runtime-cu12==12.4.127 (from torch)
  Using cached https://mirrors.aliyun.com/pypi/packages/ea/27/1795d86fe88ef397885f2e580ac37628ed058a92ed2c39dc8eac3adf0619/nvidia_cuda_runtime_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (883 kB)
Collecting nvidia-cuda-cupti-cu12==12.4.127 (from torch)
  Using cached https://mirrors.aliyun.com/pypi/packages/67/42/f4f60238e8194a3106d06a058d494b18e006c10bb2b915655bd9f6ea4cb1/nvidia_cuda_cupti_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (13.8 MB)
Collecting nvidia-cudnn-cu12==9.1.0.70 (from torch)
  Using cached https://mirrors.aliyun.com/pypi/packages/9f/fd/713452cd72343f682b1c7b9321e23829f00b842ceaedcda96e742ea0b0b3/nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl (664.8 MB)
Collecting nvidia-cublas-cu12==12.4.5.8 (from torch)
  Using cached https://mirrors.aliyun.com/pypi/packages/ae/71/1c91302526c45ab494c23f61c7a84aa568b8c1f9d196efa5993957faf906/nvidia_cublas_cu12-12.4.5.8-py3-none-manylinux2014_x86_64.whl (363.4 MB)
Collecting nvidia-cufft-cu12==11.2.1.3 (from torch)
  Downloading https://mirrors.aliyun.com/pypi/packages/27/94/3266821f65b92b3138631e9c8e7fe1fb513804ac934485a8d05776e1dd43/nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_x86_64.whl (211.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 211.5/211.5 MB 5.9 MB/s eta 0:00:00
Collecting nvidia-curand-cu12==10.3.5.147 (from torch)
  Downloading https://mirrors.aliyun.com/pypi/packages/8a/6d/44ad094874c6f1b9c654f8ed939590bdc408349f137f9b98a3a23ccec411/nvidia_curand_cu12-10.3.5.147-py3-none-manylinux2014_x86_64.whl (56.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.3/56.3 MB 5.9 MB/s eta 0:00:00
Collecting nvidia-cusolver-cu12==11.6.1.9 (from torch)
  Downloading https://mirrors.aliyun.com/pypi/packages/3a/e1/5b9089a4b2a4790dfdea8b3a006052cfecff58139d5a4e34cb1a51df8d6f/nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_x86_64.whl (127.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 127.9/127.9 MB 6.0 MB/s eta 0:00:00
Collecting nvidia-cusparse-cu12==12.3.1.170 (from torch)
  Downloading https://mirrors.aliyun.com/pypi/packages/db/f7/97a9ea26ed4bbbfc2d470994b8b4f338ef663be97b8f677519ac195e113d/nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_x86_64.whl (207.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 207.5/207.5 MB 5.9 MB/s eta 0:00:00
Collecting nvidia-cusparselt-cu12==0.6.2 (from torch)
  Downloading https://mirrors.aliyun.com/pypi/packages/78/a8/bcbb63b53a4b1234feeafb65544ee55495e1bb37ec31b999b963cbccfd1d/nvidia_cusparselt_cu12-0.6.2-py3-none-manylinux2014_x86_64.whl (150.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 150.1/150.1 MB 6.0 MB/s eta 0:00:00
Collecting nvidia-nccl-cu12==2.21.5 (from torch)
  Downloading https://mirrors.aliyun.com/pypi/packages/df/99/12cd266d6233f47d00daf3a72739872bdc10267d0383508b0b9c84a18bb6/nvidia_nccl_cu12-2.21.5-py3-none-manylinux2014_x86_64.whl (188.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 188.7/188.7 MB 5.9 MB/s eta 0:00:00
Collecting nvidia-nvtx-cu12==12.4.127 (from torch)
  Downloading https://mirrors.aliyun.com/pypi/packages/87/20/199b8713428322a2f22b722c62b8cc278cc53dffa9705d744484b5035ee9/nvidia_nvtx_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (99 kB)
Collecting nvidia-nvjitlink-cu12==12.4.127 (from torch)
  Downloading https://mirrors.aliyun.com/pypi/packages/ff/ff/847841bacfbefc97a00036e0fce5a0f086b640756dc38caea5e1bb002655/nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (21.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.1/21.1 MB 6.5 MB/s eta 0:00:00
Collecting triton==3.2.0 (from torch)
  Downloading https://mirrors.aliyun.com/pypi/packages/01/65/3ffa90e158a2c82f0716eee8d26a725d241549b7d7aaf7e4f44ac03ebd89/triton-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (253.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 253.1/253.1 MB 5.9 MB/s eta 0:00:00
Collecting sympy==1.13.1 (from torch)
  Downloading https://mirrors.aliyun.com/pypi/packages/b2/fe/81695a1aa331a842b582453b605175f419fe8540355886031328089d840a/sympy-1.13.1-py3-none-any.whl (6.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.2/6.2 MB 8.4 MB/s eta 0:00:00
Collecting mpmath<1.4,>=1.1.0 (from sympy==1.13.1->torch)
  Downloading https://mirrors.aliyun.com/pypi/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl (536 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 9.8 MB/s eta 0:00:00
Collecting numpy (from torchvision)
  Downloading https://mirrors.aliyun.com/pypi/packages/e9/88/3870cfa9bef4dffb3a326507f430e6007eeac258ebeef6b76fc542aef66d/numpy-2.2.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.4/16.4 MB 6.5 MB/s eta 0:00:00
Collecting pillow!=8.3.*,>=5.3.0 (from torchvision)
  Downloading https://mirrors.aliyun.com/pypi/packages/02/80/79f99b714f0fc25f6a8499ecfd1f810df12aec170ea1e32a4f75746051ce/pillow-11.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.4/4.4 MB 9.6 MB/s eta 0:00:00
Collecting MarkupSafe>=2.0 (from jinja2->torch)
  Downloading https://mirrors.aliyun.com/pypi/packages/22/35/137da042dfb4720b638d2937c38a9c2df83fe32d20e8c8f3185dbfef05f7/MarkupSafe-3.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (20 kB)
Installing collected packages: triton, nvidia-cusparselt-cu12, mpmath, typing-extensions, sympy, pillow, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, numpy, networkx, MarkupSafe, fsspec, filelock, nvidia-cusparse-cu12, nvidia-cudnn-cu12, jinja2, nvidia-cusolver-cu12, torch, torchvision, torchaudio
Successfully installed MarkupSafe-3.0.2 filelock-3.17.0 fsspec-2025.2.0 jinja2-3.1.5 mpmath-1.3.0 networkx-3.4.2 numpy-2.2.3 nvidia-cublas-cu12-12.4.5.8 nvidia-cuda-cupti-cu12-12.4.127 nvidia-cuda-nvrtc-cu12-12.4.127 nvidia-cuda-runtime-cu12-12.4.127 nvidia-cudnn-cu12-9.1.0.70 nvidia-cufft-cu12-11.2.1.3 nvidia-curand-cu12-10.3.5.147 nvidia-cusolver-cu12-11.6.1.9 nvidia-cusparse-cu12-12.3.1.170 nvidia-cusparselt-cu12-0.6.2 nvidia-nccl-cu12-2.21.5 nvidia-nvjitlink-cu12-12.4.127 nvidia-nvtx-cu12-12.4.127 pillow-11.1.0 sympy-1.13.1 torch-2.6.0 torchaudio-2.6.0 torchvision-0.21.0 triton-3.2.0 typing-extensions-4.12.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
(deepseek_env) root@ydpslave141:~#

3.6.VLLM安装

https://github.com/vllm-project/vllm

https://docs.vllm.ai/en/latest/getting_started/installation/gpu/index.html

使用下面命令安装vllm:

pip install vllm

安装中如果报错: 没有xgrammar。则可以手动编译xgrammar：

https://github.com/mlc-ai/xgrammar

https://xgrammar.mlc.ai/docs/start/install

编译xgrammar时，如果提示需要dlpack，则需要安装dlpack:

https://github.com/dmlc/dlpack

# 安装编译依赖
sudo yum install -y cmake git gcc-c++

# 克隆 DLPack 仓库
git clone --recursive https://github.com/dmlc/dlpack.git
cd dlpack

# 创建构建目录并编译
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr/local ..
make
sudo make install  # 安装到系统目录

ls /usr/local/include/dlpack/dlpack.h  # 验证

dlpack编译完成后，再编译xgrammar，完成后再安装vllm

安装：

(deepseek_py39) root@ydpslave141:/home/81097864/vllm# pip install .

============================================

四、Deepseek安装
4.1.deepseek下载
需要梯子进行科学上网。

ds下载时要注意，使用不同的工具构建大模型服务，要在不同的平台下载模型文件。

如果使用的是ollama，则需要去ollama平台下载模型：https://ollama.com/search

可以参考之前的文章：https://blog.csdn.net/wuxiaolongah/article/details/145500407

本次安装是参考Deepseek的Github：https://github.com/deepseek-ai/DeepSeek-R1，使用vllm做大模型服务，所以从Github页面链接下载：🤗 HuggingFace

下载过程需要注意的是，HuggingFace平台的模型不是一个文件，是将一个大模型文件切割成了多个小的模型文件。下载时不要一次性下载整个git仓库，可以先将仓库中的非模型文件clone到本地，再通过wget一个一个下载小的模型文件，下载后重命名后移动到本地仓库中。

使用第一个命令是clone整个仓库，加上GIT_LFS_SKIP_SMUDGE=1的命令是下载整个仓库但是不包括模型文件。

4.2.deepseek启动

模型准备完成后就可以启动vllm了：

看到INFO: Uvicorn running on http://0.0.0.0:18090 (Press CTRL+C to quit)

表示服务启动成功，可以通过多种方式访问deepseek了.

4.3.deepseek访问
vllm的相关接口参数可以参考官方：https://docs.vllm.ai/en/latest/getting_started/quickstart.html

方式一：（推荐）
使用chatbox

本地下载

安装后配置：按照vllm启动时的相关参数配置。vllm提供了和OpenAI兼容的API

然后开启新对话即可聊天。

注意这里的红色部分参数一定要按照启动参数配置。

方式二：

登陆141机器, 使用curl方式调用

curl http://1xxxxx:18090/v1/chat/completions -H "Content-Type: application/json"  -d '{   "model": "DeepSeek-R1-Distill-Qwen-32B",   "messages": [    {"role": "system", "content": "你是个有用的助手."}, {"role": "user", "content": "未来中美博弈谁会赢"}]}'

方式三：

本地内网使用postman调用

其他：



#动态监控显卡状态
(base) root@ydpslave141:/home/81097864/gup-driver# watch -n 1 nvidia-smi
Every 1.0s: nvidia-smi                                                                                                                             Fri Feb 14 16:14:54 2025

Fri Feb 14 16:14:54 2025
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.19.01    Driver Version: 465.19.01    CUDA Version: 11.3     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA Graphics...  Off  | 00000000:65:00.0 Off |                    0 |
| N/A   39C    P0    43W / 300W |      3MiB / 81251MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

阅读剩余

作者：SE_You

链接：https://www.cnesa.cn/4006.html

文章版权归作者所有，未经允许请勿转载。

THE END

这样配置防火墙双机热备，网络很难出现故障

<<上一篇

【WALL1600 下一代防火墙】NGFW防火墙无法正常启动，停在启动阶段出现报错

下一篇>>