(大语言模型) transformers 目前支持两种量化方式：bitsandbytes 和 autogptq

一、报错信息

bin C:\Users\win10\anaconda3\envs\qlora\Lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so
False
C:\Users\win10\anaconda3\envs\qlora\Lib\site-packages\bitsandbytes\cuda_setup\main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/Users/win10/anaconda3/envs/qlora/lib'), WindowsPath('C')}warn(msg)
C:\Users\win10\anaconda3\envs\qlora\Lib\site-packages\bitsandbytes\cuda_setup\main.py:149: UserWarning: C:\Users\win10\anaconda3\envs\qlora did 
not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
C:\Users\win10\anaconda3\envs\qlora\Lib\site-packages\bitsandbytes\cuda_setup\main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/usr/local/cuda/lib64')}warn(msg)
CUDA SETUP: WARNING! libcuda.so not found! Do you have a CUDA driver installed? If you are on a cluster, make sure you are on a CUDA machine!   
C:\Users\win10\anaconda3\envs\qlora\Lib\site-packages\bitsandbytes\cuda_setup\main.py:149: UserWarning: WARNING: No libcudart.so found! Install 
CUDA or the cudatoolkit package (anaconda)!warn(msg)
C:\Users\win10\anaconda3\envs\qlora\Lib\site-packages\bitsandbytes\cuda_setup\main.py:149: UserWarning: WARNING: No GPU detected! Check your CUDA paths. Proceeding to load CPU-only library...warn(msg)
CUDA SETUP: Loading binary C:\Users\win10\anaconda3\envs\qlora\Lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so...
argument of type 'WindowsPath' is not iterable
CUDA SETUP: Problem: The main issue seems to be that the main CUDA library was not detected.
CUDA SETUP: Solution 1): Your paths are probably not up-to-date. You can update them via: sudo ldconfig.
CUDA SETUP: Solution 2): If you do not have sudo rights, you can do the following:
CUDA SETUP: Solution 2a): Find the cuda library via: find / -name libcuda.so 2>/dev/null
CUDA SETUP: Solution 2b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_2a   
CUDA SETUP: Solution 2c): For a permanent solution add the export from 2b into your .bashrc file, located at ~/.bashrc

二、问题分析

如果要在 Windows 平台上开启量化 LoRA（QLoRA），需要安装预编译的 bitsandbytes 库, 支持 CUDA 11.1 到 12.2, 根据您的 CUDA 版本情况选择适合的发布版本。

Release Wheels · jllllll/bitsandbytes-windows-webui · GitHub

三、解决方案

（1）直接安装预编译的库

pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.2.post2-py3-none-win_amd64.whl

（2）安装 bitsandbytes-windows

pip uninstall bitsandbytes
pip install bitsandbytes-windows