快速体验fastllm安装部署并支持AMD ROCm推理加速

news/2024/9/23 3:55:57/

序言

fastllm是纯c++实现，无第三方依赖的高性能大模型推理库。

本文以国产海光DCU为例，在AMD ROCm平台下编译部署fastllm以实现LLMs模型推理加速。

测试平台：曙光超算互联网平台SCNet

GPU/DCU：异构加速卡AI 显存64GB PCIE（基于ROCm平台的GPU）

服务器的详细配置，请参考：曙光超算互联网平台SCNet之国产异构加速卡DCU

一、参考资料

fastllm 代码仓库：https://github.com/lcpu-club/fastllm-rocm

二、快速开始

fastllm_18">1. fastllm功能概述

  • 🚀 纯c++实现，便于跨平台移植，可以在安卓上直接编译。
  • 🚀 ARM平台支持NEON指令集加速，X86平台支持AVX指令集加速，NVIDIA平台支持CUDA加速。
  • 🚀 支持浮点模型（FP32), 半精度模型(FP16), 量化模型(INT8, INT4) 加速。
  • 🚀 支持多卡部署，支持GPU + CPU混合部署。
  • 🚀 支持Batch速度优化。
  • 🚀 支持并发计算时动态拼Batch。
  • 🚀 支持流式输出，很方便实现打字机效果。
  • 🚀 支持python调用。
  • 🚀 前后端分离设计，便于支持新的计算设备。
  • 🚀 目前支持ChatGLM模型，各种LLAMA模型(ALPACA, VICUNA等)，BAICHUAN模型，MOSS模型。

2. 测试环境

root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/build# hipconfig
HIP version  : 5.7.24164== hipconfig
HIP_PATH     : /opt/dtk/hip
ROCM_PATH    : /opt/dtk
HIP_COMPILER : clang
HIP_PLATFORM : amd
HIP_RUNTIME  : rocclr
CPP_CONFIG   :  -D__HIP_PLATFORM_HCC__= -D__HIP_PLATFORM_AMD__= -I"/opt/dtk/hip/include" -I"/opt/dtk/llvm/bin/../lib/clang/15.0.0" -I/opt/dtk/hsa/include== hip-clang
HSA_PATH         : /opt/dtk/hsa
HIP_CLANG_PATH   : /opt/dtk/llvm/bin
clang version 15.0.0
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/dtk/llvm/bin
AOMP-15.0-61 (http://github.com/ROCm-Developer-Tools/aomp):Source ID:15.0-61-595b0d8133fafef5742f7d39f8e6a07b31afff56LLVM version 15.0.0gitOptimized build.Default target: x86_64-unknown-linux-gnuHost CPU: (unknown)Registered Targets:amdgcn - AMD GCN GPUsr600   - AMD GPUs HD2XXX-HD6XXXx86    - 32-bit X86: Pentium-Pro and abovex86-64 - 64-bit X86: EM64T and AMD64
hip-clang-cxxflags :  -std=c++11 -isystem /opt/dtk-24.04/llvm/lib/clang/15.0.0/include/.. -isystem /opt/dtk-24.04/llvm/lib/clang/15.0.0/include/cuda_wrappers -isystem /opt/dtk/hsa/include -isystem /opt/dtk/hip/include -msram-ecc -msram-ecc -msram-ecc -O3
hip-clang-ldflags  : --driver-mode=g++ -L"/opt/dtk/hip/lib" -L"/opt/dtk/lib" -O3 -lgcc_s -lgcc -lpthread -lm -lrt=== Environment Variables
PATH=/usr/local/bin:/opt/mpi/bin:/opt/hwloc/bin/:/opt/cmake/bin/:/opt/conda/bin:/opt/conda/bin:/opt/dtk/bin:/opt/dtk/llvm/bin:/opt/dtk/hip/bin:/opt/dtk/hip/bin/hipify:/opt/hyhal/bin:/opt/mpi/bin:/opt/hwloc/bin/:/opt/cmake/bin/:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/conda/bin
LD_LIBRARY_PATH=/usr/local/lib/:/usr/local/lib64/:/opt/mpi/lib:/opt/hwloc/lib:/opt/dtk/hip/lib:/opt/dtk/llvm/lib:/opt/dtk/lib:/opt/dtk/lib64:/opt/hyhal/lib:/opt/hyhal/lib64:/opt/dtk-24.04/hip/lib:/opt/dtk-24.04/llvm/lib:/opt/dtk-24.04/lib:/opt/dtk-24.04/lib64:/opt/hyhal/lib:/opt/hyhal/lib64:/usr/local/lib/:/usr/local/lib64/:/opt/mpi/lib:/opt/hwloc/lib:
HIP_PATH=/opt/dtk/hip== Linux Kernel
Hostname     : notebook-1823641624653922306-scnlbe5oi5-42808
Linux notebook-1823641624653922306-scnlbe5oi5-42808 3.10.0-957.el7.x86_64 #1 SMP Fri Oct 25 22:34:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

3. 安装CLBlast

CLBlast: Building and installing

4. cmake编译

# 下载源码
git clone https://github.com/lcpu-club/fastllm-rocm.gitcd fastllm
mkdir build
cd build# 设置环境变量
export ROCM_PATH="/opt/dtk"
export CLBlast_DIR="/usr/lib/x86_64-linux-gnu/cmake/CLBlast"export CXXFLAGS="-I${ROCM_PATH}/include -Wno-pass-failed -D__HIP_PLATFORM_AMD__"# cmake
cmake .. -DUSE_ROCM=ON -DCMAKE_CXX_COMPILER=/opt/dtk/llvm/bin/clang++

root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastl
lm-rocm/build# cmake .. -DUSE_ROCM=ON -DCMAKE_CXX_COMPILER=/opt/dtk/llvm/bin/clang++
-- The CXX compiler identification is Clang 15.0.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/dtk/llvm/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- USE_CUDA: OFF
-- USE_ROCM: ON
-- PYTHON_API: OFF
-- USE_SENTENCEPIECE: OFF
-- CMAKE_CXX_FLAGS-I/opt/dtk/include -Wno-pass-failed -D__HIP_PLATFORM_AMD__ -pthread --std=c++17 -O3 -march=native
-- The HIP compiler identification is Clang 15.0.0
-- Detecting HIP compiler ABI info
-- Detecting HIP compiler ABI info - done
-- Check for working HIP compiler: /opt/dtk/llvm/bin/clang++ - skipped
-- Detecting HIP compile features
-- Detecting HIP compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS - Success
-- System architecture is x86_64
-- Configuring done (5.1s)
-- Generating done (0.6s)
-- Build files have been written to: /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/build

5. make编译

单线程编译

root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/bu
ild# make -j1
[  2%] Building CXX object CMakeFiles/fastllm.dir/src/fastllm.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:46: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:32: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:18: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
9 warnings generated.
[  4%] Building CXX object CMakeFiles/fastllm.dir/src/device.cpp.o
[  6%] Building CXX object CMakeFiles/fastllm.dir/src/model.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
1 warning generated.
[  8%] Building CXX object CMakeFiles/fastllm.dir/src/executor.cpp.o
[ 11%] Building CXX object CMakeFiles/fastllm.dir/src/devices/cpu/cpudevice.cpp.o
[ 13%] Building CXX object CMakeFiles/fastllm.dir/src/devices/cpu/cpudevicebatch.cpp.o
[ 15%] Building CXX object CMakeFiles/fastllm.dir/src/models/chatglm.cpp.o
[ 17%] Building CXX object CMakeFiles/fastllm.dir/src/models/moss.cpp.o
[ 20%] Building CXX object CMakeFiles/fastllm.dir/src/models/llama.cpp.o
[ 22%] Building CXX object CMakeFiles/fastllm.dir/src/models/qwen.cpp.o
[ 24%] Building CXX object CMakeFiles/fastllm.dir/src/models/basellm.cpp.o
[ 26%] Building CXX object CMakeFiles/fastllm.dir/src/models/glm.cpp.o
[ 28%] Building CXX object CMakeFiles/fastllm.dir/src/devices/rocm/rocmdevice.cpp.o
[ 31%] Building CXX object CMakeFiles/fastllm.dir/src/devices/rocm/rocmdevicebatch.cpp.o
[ 33%] Building HIP object CMakeFiles/fastllm.dir/src/devices/rocm/fastllm-rocm.hip.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for gfx906.
4 warnings generated when compiling for gfx926.
4 warnings generated when compiling for gfx928.
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for host.
[ 33%] Built target fastllm
[ 35%] Building CXX object CMakeFiles/main.dir/main.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/main.cpp:1:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/main.cpp:64:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[ 37%] Linking HIP executable main
[ 37%] Built target main
[ 40%] Building CXX object CMakeFiles/quant.dir/tools/src/quant.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/quant.cpp:6:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/quant.cpp:49:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[ 42%] Linking HIP executable quant
[ 42%] Built target quant
[ 44%] Building CXX object CMakeFiles/testOps.dir/test/ops/cppOps.cpp.o
[ 46%] Linking HIP executable testOps
[ 46%] Built target testOps
[ 48%] Building CXX object CMakeFiles/webui.dir/example/webui/webui.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/webui/webui.cpp:4:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/webui/webui.cpp:76:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[ 51%] Linking HIP executable webui
[ 51%] Built target webui
[ 53%] Building CXX object CMakeFiles/benchmark.dir/example/benchmark/benchmark.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/benchmark/benchmark.cpp:5:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/benchmark/benchmark.cpp:82:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[ 55%] Linking HIP executable benchmark
[ 55%] Built target benchmark
[ 57%] Building CXX object CMakeFiles/apiserver.dir/example/apiserver/apiserver.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/apiserver.cpp:4:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/apiserver.cpp:231:8: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested here
struct WorkQueue {^
1 warning generated.
[ 60%] Building CXX object CMakeFiles/apiserver.dir/example/apiserver/json11.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:251:58: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonString(string &&value)      : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:259:62: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonArray(Json::array &&value)      : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:267:64: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonObject(Json::object &&value)      : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:309:76: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(string &&value)             : m_ptr(make_shared<JsonString>(move(value))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:312:75: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(Json::array &&values)       : m_ptr(make_shared<JsonArray>(move(values))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:314:76: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(Json::object &&values)      : m_ptr(make_shared<JsonObject>(move(values))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:412:29: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]return fail(move(msg), Json());^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value)      : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:251:52: note: in instantiation of member function 'json11::Value<json11::Json::STRING, std::basic_string<char>>::Value' requested hereexplicit JsonString(string &&value)      : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value)      : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:259:56: note: in instantiation of member function 'json11::Value<json11::Json::ARRAY, std::vector<json11::Json>>::Value' requested hereexplicit JsonArray(Json::array &&value)      : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value)      : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:267:58: note: in instantiation of member function 'json11::Value<json11::Json::OBJECT, std::map<std::basic_string<char>, json11::Json>>::Value' requested hereexplicit JsonObject(Json::object &&value)      : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value)      : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:272:22: note: in instantiation of member function 'json11::Value<json11::Json::NUL, json11::NullStruct>::Value' requested hereJsonNull() : Value({}) {}^
11 warnings generated.
[ 62%] Linking HIP executable apiserver
[ 62%] Built target apiserver
[ 64%] Building CXX object CMakeFiles/fastllm_tools.dir/src/fastllm.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:46: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:32: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:18: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
9 warnings generated.
[ 66%] Building CXX object CMakeFiles/fastllm_tools.dir/src/device.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/model.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
1 warning generated.
[ 71%] Building CXX object CMakeFiles/fastllm_tools.dir/src/executor.cpp.o
[ 73%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/cpu/cpudevice.cpp.o
[ 75%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/cpu/cpudevicebatch.cpp.o
[ 77%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/chatglm.cpp.o
[ 80%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/moss.cpp.o
[ 82%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/llama.cpp.o
[ 84%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/qwen.cpp.o
[ 86%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/basellm.cpp.o
[ 88%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/glm.cpp.o
[ 91%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/rocm/rocmdevice.cpp.o
[ 93%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/rocm/rocmdevicebatch.cpp.o
[ 95%] Building HIP object CMakeFiles/fastllm_tools.dir/src/devices/rocm/fastllm-rocm.hip.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9:/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip                                                    ^~~~~~~~~~~
:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for gfx926.
4 warnings generated when compiling for gfx928.
4 warnings generated when compiling for gfx906.
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for host.
[ 97%] Building CXX object CMakeFiles/fastllm_tools.dir/tools/src/pytools.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:78:42: warning: 'make_config' has C-linkage specified, but returns user-defined type 'fastllm::GenerationConfig' which is incompatible with C [-Wreturn-type-c-linkage]DLL_EXPORT fastllm::GenerationConfig make_config(int max_length, bool do_sample, float top_p, int top_k,^
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:5:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:95:29: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested heremodels.models[id] = fastllm::CreateLLMModelFromFile(path);^
2 warnings generated.
[100%] Linking HIP shared library libfastllm_tools.so
[100%] Built target fastllm_tools

多线程编译

root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastl
lm-rocm/build# make -j${nproc}
[  6%] Building CXX object CMakeFiles/fastllm.dir/src/fastllm.cpp.o
[ 17%] Building CXX object CMakeFiles/fastllm.dir/src/device.cpp.o
[  6%] Building CXX object CMakeFiles/fastllm.dir/src/model.cpp.o
[ 46%] Building CXX object CMakeFiles/fastllm.dir/src/executor.cpp.o
[ 46%] Building CXX object CMakeFiles/fastllm_tools.dir/src/fastllm.cpp.o
[ 48%] Building CXX object CMakeFiles/fastllm.dir/src/devices/cpu/cpudevice.cpp.o
[ 53%] Building CXX object CMakeFiles/fastllm.dir/src/models/chatglm.cpp.o
[ 55%] Building CXX object CMakeFiles/fastllm_tools.dir/src/device.cpp.o
[ 66%] Building CXX object CMakeFiles/fastllm.dir/src/devices/cpu/cpudevicebatch.cpp.o
[ 66%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/rocm/rocmdevicebatch.cpp.o
[ 68%] Building HIP object CMakeFiles/fastllm_tools.dir/src/devices/rocm/fastllm-rocm.hip.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/model.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/moss.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/cpu/cpudevice.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/basellm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/executor.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/cpu/cpudevicebatch.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/llama.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/glm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/qwen.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/chatglm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/moss.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/qwen.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/llama.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/devices/rocm/rocmdevice.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/devices/rocm/rocmdevicebatch.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/rocm/rocmdevice.cpp.o
[ 68%] Building HIP object CMakeFiles/fastllm.dir/src/devices/rocm/fastllm-rocm.hip.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/glm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/basellm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/tools/src/pytools.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:78:42: warning: 'make_config' has C-linkage specified, but returns user-defined type 'fastllm::GenerationConfig' which is incompatible with C [-Wreturn-type-c-linkage]DLL_EXPORT fastllm::GenerationConfig make_config(int max_length, bool do_sample, float top_p, int top_k,^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:5:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:95:29: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested heremodels.models[id] = fastllm::CreateLLMModelFromFile(path);^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:46: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:32: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:18: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:46: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:32: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:18: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
2 warnings generated.
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
1 warning generated.
4 warnings generated when compiling for gfx928.
1 warning generated.
4 warnings generated when compiling for gfx926.
4 warnings generated when compiling for gfx906.
4 warnings generated when compiling for gfx906.
4 warnings generated when compiling for gfx926.
4 warnings generated when compiling for gfx928.
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^  ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~   ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~   ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for host.
4 warnings generated when compiling for host.
9 warnings generated.
9 warnings generated.
[ 71%] Linking HIP shared library libfastllm_tools.so
[ 71%] Built target fastllm_tools
[ 71%] Built target fastllm
[ 77%] Building CXX object CMakeFiles/testOps.dir/test/ops/cppOps.cpp.o
[ 77%] Building CXX object CMakeFiles/benchmark.dir/example/benchmark/benchmark.cpp.o
[ 77%] Building CXX object CMakeFiles/main.dir/main.cpp.o
[ 86%] Building CXX object CMakeFiles/webui.dir/example/webui/webui.cpp.o
[ 86%] Building CXX object CMakeFiles/quant.dir/tools/src/quant.cpp.o
[ 86%] Building CXX object CMakeFiles/apiserver.dir/example/apiserver/apiserver.cpp.o
[ 86%] Building CXX object CMakeFiles/apiserver.dir/example/apiserver/json11.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:251:58: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonString(string &&value)      : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:259:62: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonArray(Json::array &&value)      : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:267:64: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonObject(Json::object &&value)      : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:309:76: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(string &&value)             : m_ptr(make_shared<JsonString>(move(value))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:312:75: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(Json::array &&values)       : m_ptr(make_shared<JsonArray>(move(values))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:314:76: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(Json::object &&values)      : m_ptr(make_shared<JsonObject>(move(values))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:412:29: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]return fail(move(msg), Json());^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value)      : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:251:52: note: in instantiation of member function 'json11::Value<json11::Json::STRING, std::basic_string<char>>::Value' requested hereexplicit JsonString(string &&value)      : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value)      : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:259:56: note: in instantiation of member function 'json11::Value<json11::Json::ARRAY, std::vector<json11::Json>>::Value' requested hereexplicit JsonArray(Json::array &&value)      : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value)      : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:267:58: note: in instantiation of member function 'json11::Value<json11::Json::OBJECT, std::map<std::basic_string<char>, json11::Json>>::Value' requested hereexplicit JsonObject(Json::object &&value)      : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value)      : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:272:22: note: in instantiation of member function 'json11::Value<json11::Json::NUL, json11::NullStruct>::Value' requested hereJsonNull() : Value({}) {}^
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/quant.cpp:6:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/quant.cpp:49:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/main.cpp:1:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/main.cpp:64:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
[ 88%] Linking HIP executable testOps
1 warning generated.
[ 91%] Linking HIP executable quant
[ 91%] Built target testOps
1 warning generated.
[ 93%] Linking HIP executable main
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/benchmark/benchmark.cpp:5:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/benchmark/benchmark.cpp:82:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
[ 93%] Built target quant
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/apiserver.cpp:4:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/apiserver.cpp:231:8: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested here
struct WorkQueue {^
[ 93%] Built target main
1 warning generated.
[ 95%] Linking HIP executable benchmark
[ 95%] Built target benchmark
11 warnings generated.
1 warning generated.
[ 97%] Linking HIP executable apiserver
[ 97%] Built target apiserver
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/webui/webui.cpp:4:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/webui/webui.cpp:76:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[100%] Linking HIP executable webui
[100%] Built target webui

6. 运行demo

准备模型，以 huangyuyang/chatglm2-6b-int4.flm 为例。

# 命令行聊天程序
./main -p chatglm2-6b-int4.flm

root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastl
ild# ./main -p chatglm2-6b-int4.flm
AVX: ON
AVX2: ON
AARCH64: OFF
Neon FP16: OFF
Neon DOT: OFF
Load (200 / 200)
Warmup...
Launch params (512, 1, 1) are larger than launch bounds (256) for kernel _Z26FastllmRMSNormKerne512EEvPfS0_S0_iif please add __launch_bounds__ to kernel define or use --gpu-max-threads-per-blole program !
finish.
欢迎使用 chatglm 模型. 输入内容对话，reset清空历史记录，stop退出程序.
用户: hi
chatglm: Hello! How can I assist you today?
用户: 中国深圳有哪些旅游景点
chatglm: 深圳是一个美丽的城市,有很多值得一游的景点。以下是一些深圳的旅游景点:1. 深圳湾公园:是一座大型公园,位于深圳湾海滨,可以欣赏到美丽的海景和城市景观。2. 华侨城:这是一个综合性的文化城,里面有很多娱乐设施和景点,如大型主题公园、音乐厅和博物馆等。3. 深圳会展中心:这是一个现代化的展览中心,可以举办各种展览和活动。4. 东部华侨城:这是一个以海洋文化为主题的大型综合性景区,里面有很多海洋生物和各种娱乐项目。5. 深圳中心书城:这是一个大型的书店,里面有很多图书和文化产品。6. 深圳动物园:这是一个大型的动物园,可以了解到各种动物和它们的生活习性。7. 大梅沙海滨公园:这是一个美丽的海滨公园,里面有很多户外活动设施和景观。8. 深圳技术馆:这是一个展示科技和文化的博物馆,可以了解到各种科技发展的趋势和文化。9. 深圳音乐厅:这是一个现代化的音乐厅,可以举办各种音乐演出和活动。10. 深圳海底世界:这是一个大型的海洋生物博物馆,可以了解到各种海洋生物和它们的生活习性。

运行时的资源占用情况

在这里插入图片描述

在这里插入图片描述

三、FAQ

Q：#error ("Must define exactly one of __HIP_PLATFORM_AMD__ or __HIP_PLATFORM_NVIDIA__"

Must define exactly one of __HIP_PLATFORM_AMD__ or __HIP_PLATFORM_NVIDIA__ Error #1789

/opt/dtk/include/hip/hip_vector_types.h:38:2: error: #error ("Must define exactly one of __HIP_PLATFORM_AMD__ or __HIP_PLATFORM_NVIDIA__");38 | #error("Must define exactly one of __HIP_PLATFORM_AMD__ or __HIP_PLATFORM_NVIDIA__");|  ^~~~~
In file included from /opt/dtk/include/hip/hip_runtime.h:113,from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/devices/rocm/fastllm-rocm.hiph:2,from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:38:
/opt/dtk/include/hip/hip_runtime_api.h:8549:61: error: 'hipHostMallocDefault' was not declared in this scope8549 |                                        unsigned int flags = hipHostMallocDefault) {|                                                             ^~~~~~~~~~~~~~~~~~~~
/opt/dtk/include/hip/hip_runtime_api.h:8568:61: error: 'hipMemAttachGlobal' was not declared in this scope8568 |                                        unsigned int flags = hipMemAttachGlobal) {|                                                             ^~~~~~~~~~~~~~~~~~
make[2]: *** [CMakeFiles/fastllm.dir/build.make:76: CMakeFiles/fastllm.dir/src/fastllm.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:97: CMakeFiles/fastllm.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

错误原因：未指定 D__HIP_PLATFORM_AMD__。

解决方法：

# 设置环境变量
export CXXFLAGS="-I${ROCM_PATH}/include -Wno-pass-failed -D__HIP_PLATFORM_AMD__"

Q：error: use of undeclared identifier 'CUR_DEVICE' contextLayer.ToDevice(CUR_DEVICE)

/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/models/chatglm.cpp:382:31: error: use of undeclared identifier 'CUR_DEVICE'hiddenStates.ToDevice(CUR_DEVICE);^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/models/chatglm.cpp:518:34: error: use of undeclared identifier 'CUR_DEVICE'pastKey.ToDevice(CUR_DEVICE);^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/models/chatglm.cpp:519:36: error: use of undeclared identifier 'CUR_DEVICE'pastValue.ToDevice(CUR_DEVICE);^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/models/chatglm.cpp:668:43: error: use of undeclared identifier 'CUR_DEVICE'contextLayer.ToDevice(CUR_DEVICE);^
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
1 error generated.
make[2]: *** [CMakeFiles/fastllm.dir/build.make:90: CMakeFiles/fastllm.dir/src/device.cpp.o] Error 1

错误原因：未指定 DUSE_ROCM。

解决方法：

cmake .. -DUSE_ROCM=ON -DCMAKE_CXX_COMPILER=/opt/dtk/llvm/bin/clang++

http://www.ppmy.cn/news/1511077.html

相关文章

高级java每日一道面试题-2024年8月15日-设计模式篇-设计模式与面向对象原则的关系是什么？

高级java每日一道面试题-2024年8月15日-设计模式篇-设计模式与面向对象原则的关系是什么？

如果有遗漏,评论区告诉我进行补充 面试官: 设计模式与面向对象原则的关系是什么&#xff1f; 我回答: 在设计模式与面向对象原则的关系中&#xff0c;两者紧密相连且相互促进。面向对象的原则为设计模式的形成提供了理论基础和指导思想&#xff0c;而设计模式则是这些原则在…
阅读更多...
大型饲料粉碎机设备-饲料加工新选择

大型饲料粉碎机设备-饲料加工新选择

在追求效率与品质的现代养殖业中&#xff0c;永兴牌大型饲料粉碎机设备是您不可或缺的得力助手。采用多级粉碎技术与高耐磨锤片&#xff0c;轻松应对各种硬质及黏原料&#xff0c;实现细化&#xff0c;保留营养成分。智能化控制一键启动&#xff0c;自动调节&#xff0c;让饲料…
阅读更多...
悬浮球 可拖拽-支持鼠标和触控

悬浮球 可拖拽-支持鼠标和触控

悬浮球 可拖拽-支持鼠标和触控–react写法 直接上代码&#xff0c;js代码片段如下 import React, { useRef, useState, useEffect } from react; import { useHistory, withRouter } from react-router-dom; import ./index.css;let firstTime 0, lastTime 0; const Floati…
阅读更多...
(寄中寄) docker destop 的wsl问题——记一次在win10虚拟机的上的启动docker

(寄中寄) docker destop 的wsl问题——记一次在win10虚拟机的上的启动docker

&#x1f3aa;&#x1f3a2;&#x1f3a1;&#x1f3a0;&#x1f921;&#x1f981; 在win10上可能可以用docker&#xff0c;不过在win10上用docker有点不可能 wsl在windows中可能需要安装&#xff0c;不过wsl在windows安装有点不可能 应该是不用安装吧 可能需要重启 然后 …
阅读更多...
指针详解（三）

指针详解（三）

目录 1. 数组名 2. 使用指针访问数组 3. 一维数组传参的本质 4. 冒泡排序 5. 二级指针 6. 指针数组 7. 指针数组模拟二维数组 1. 数组名 在使用指针访问数组的内容时&#xff0c;有这样的代码&#xff1a; int arr[10] {1,2,3,4,5,6,7,8,9,10}; int *p &arr[0];/…
阅读更多...
centos7 服务器搭建

centos7 服务器搭建

1. 查看 centos 版本 cat /etc/redhat-release CentOS Linux release 7.9.2009 (Core)2 .查看 ip地址 ip addr sudo yum install net-tools -y 3. 是否能够上网 ping www.baidu.com ping 114.114.114.114 sudo systemctl restart network 4. DNS 更新DNS配置 编辑/etc/r…
阅读更多...
平安城市/雪亮工程现状及需求分析：EasyCVR视频汇聚平台助力雪亮工程项目建设

平安城市/雪亮工程现状及需求分析：EasyCVR视频汇聚平台助力雪亮工程项目建设

一、背景现状 经过近几年的努力&#xff0c;平安城市雪亮工程建设取得了显著的成绩&#xff0c;完成了前端高清视频点位和高清卡口系统建设&#xff0c;建成了&#xff08;视频监控类&#xff09;、&#xff08;卡口类&#xff09;和&#xff08;应用类&#xff09;的平台。这…
阅读更多...
【Solidity】函数的使用

【Solidity】函数的使用

构造函数 构造函数仅在部署合约时调用一次&#xff0c;它的作用主要是初始化一些状态变量。 contract Demo {address public owner;uint public num;constructor(uint _num) {owner msg.sender;num _num;} }函数装饰器 函数装饰器可以在函数执行之前或之后插入代码逻辑&am…
阅读更多...
最新文章

Copyright @ 2022~2023