Atom-7B-Chat 本地推理
基础环境信息（wsl2安装Ubuntu22.04 + miniconda）
使用miniconda搭建环境
(base) :~$ conda create --name Llama-Chinese python=3.10
Retrieving notices: ...working... done
Channels:- defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done## Package Plan ##environment location: /home/chop/miniconda3/envs/Llama-Chineseadded / updated specs:- python=3.10The following NEW packages will be INSTALLED:_libgcc_mutex anaconda/pkgs/main/linux-64::_libgcc_mutex-0.1-main_openmp_mutex anaconda/pkgs/main/linux-64::_openmp_mutex-5.1-1_gnubzip2 anaconda/pkgs/main/linux-64::bzip2-1.0.8-h5eee18b_5ca-certificates anaconda/pkgs/main/linux-64::ca-certificates-2024.3.11-h06a4308_0ld_impl_linux-64 anaconda/pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1libffi anaconda/pkgs/main/linux-64::libffi-3.4.4-h6a678d5_0libgcc-ng anaconda/pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1libgomp anaconda/pkgs/main/linux-64::libgomp-11.2.0-h1234567_1libstdcxx-ng anaconda/pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1libuuid anaconda/pkgs/main/linux-64::libuuid-1.41.5-h5eee18b_0ncurses anaconda/pkgs/main/linux-64::ncurses-6.4-h6a678d5_0openssl anaconda/pkgs/main/linux-64::openssl-3.0.13-h7f8727e_0pip anaconda/pkgs/main/linux-64::pip-23.3.1-py310h06a4308_0python anaconda/pkgs/main/linux-64::python-3.10.14-h955ad1f_0readline anaconda/pkgs/main/linux-64::readline-8.2-h5eee18b_0setuptools anaconda/pkgs/main/linux-64::setuptools-68.2.2-py310h06a4308_0sqlite anaconda/pkgs/main/linux-64::sqlite-3.41.2-h5eee18b_0tk anaconda/pkgs/main/linux-64::tk-8.6.12-h1ccaba5_0tzdata anaconda/pkgs/main/noarch::tzdata-2024a-h04d1e81_0wheel anaconda/pkgs/main/linux-64::wheel-0.41.2-py310h06a4308_0xz anaconda/pkgs/main/linux-64::xz-5.4.6-h5eee18b_0zlib anaconda/pkgs/main/linux-64::zlib-1.2.13-h5eee18b_0Proceed ([y]/n)? yDownloading and Extracting Packages:Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate Llama-Chinese
#
# To deactivate an active environment, use
#
# $ conda deactivate(base) :~$ conda activate Llama-Chinese下载Llama-Chinese
(Llama-Chinese) :~$git init
(Llama-Chinese) :~$git clone https://github.com/LlamaFamily/Llama-Chinese.git
(Llama-Chinese) :~$ cd Llama-Chinese
(Llama-Chinese) :~/Llama-Chinese$ ls
README.md README_EN.md assets data docker docs examples inference-speed requirements.txt scripts train
安装所需要的软件包
(Llama-Chinese) :~/Llama-Chinese/Llama-Chinese$ pip install -r requirements.txt
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting torch==2.1.2 (from -r requirements.txt (line 1))Using cached https://pypi.tuna.tsinghua.edu.cn/packages/03/f1/13137340776dd5d5bcfd2574c9c6dfcc7618285035cd77240496e5c1a79b/torch-2.1.2-cp310-cp310-manylinux1_x86_64.whl (670.2 MB)
Requirement already satisfied: bitsandbytes==0.42.0 in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 2)) (0.42.0)
Requirement already satisfied: accelerate==0.27.2 in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 3)) (0.27.2)
Requirement already satisfied: numpy==1.26.4 in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 4)) (1.26.4)
Requirement already satisfied: gekko==1.0.6 in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 5)) (1.0.6)
Requirement already satisfied: pandas in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 6)) (2.2.2)
Requirement already satisfied: scipy in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 7)) (1.13.0)
Requirement already satisfied: sentencepiece==0.2.0 in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 8)) (0.2.0)
Requirement already satisfied: datasets in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 9)) (2.19.0)
Requirement already satisfied: evaluate in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 10)) (0.4.1)
Requirement already satisfied: pytest in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 11)) (8.1.1)
Requirement already satisfied: peft==0.8.2 in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 12)) (0.8.2)
Requirement already satisfied: transformers==4.39.0 in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 13)) (4.39.0)
Collecting deepspeed==0.14.0 (from -r requirements.txt (line 14))Using cached deepspeed-0.14.0-py3-none-any.whl
Requirement already satisfied: scikit-learn in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 15)) (1.4.2)
Requirement already satisfied: torchvision in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 16)) (0.17.2)
Requirement already satisfied: torchdata in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 17)) (0.7.1)
Requirement already satisfied: torchaudio in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 18)) (2.2.2)
Requirement already satisfied: tensorboard in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 19)) (2.16.2)
Requirement already satisfied: gradio in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 20)) (4.27.0)
Requirement already satisfied: packaging in /home/chop/.local/lib/python3.10/site-packages (from -r requirements.txt (line 21)) (24.0)
Requirement already satisfied: filelock in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (3.13.4)
Requirement already satisfied: typing-extensions in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (4.11.0)
Requirement already satisfied: sympy in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (1.12)
Requirement already satisfied: networkx in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (3.3)
Requirement already satisfied: jinja2 in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (3.1.3)
Requirement already satisfied: fsspec in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (2024.3.1)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (12.1.105)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (12.1.105)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (12.1.105)
Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (8.9.2.26)
Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (12.1.3.1)
Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (11.0.2.54)
Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (10.3.2.106)
Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (11.4.5.107)
Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (12.1.0.106)
Collecting nvidia-nccl-cu12==2.18.1 (from torch==2.1.2->-r requirements.txt (line 1))Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a4/05/23f8f38eec3d28e4915725b233c24d8f1a33cb6540a882f7b54be1befa02/nvidia_nccl_cu12-2.18.1-py3-none-manylinux1_x86_64.whl (209.8 MB)
Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /home/chop/.local/lib/python3.10/site-packages (from torch==2.1.2->-r requirements.txt (line 1)) (12.1.105)
Collecting triton==2.1.0 (from torch==2.1.2->-r requirements.txt (line 1))Using cached https://pypi.tuna.tsinghua.edu.cn/packages/4d/22/91a8af421c8a8902dde76e6ef3db01b258af16c53d81e8c0d0dc13900a9e/triton-2.1.0-0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (89.2 MB)
Requirement already satisfied: psutil in /home/chop/.local/lib/python3.10/site-packages (from accelerate==0.27.2->-r requirements.txt (line 3)) (5.9.8)
Collecting pyyaml (from accelerate==0.27.2->-r requirements.txt (line 3))Using cached https://pypi.tuna.tsinghua.edu.cn/packages/29/61/bf33c6c85c55bc45a29eee3195848ff2d518d84735eb0e2d8cb42e0d285e/PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)
Requirement already satisfied: huggingface-hub in /home/chop/.local/lib/python3.10/site-packages (from accelerate==0.27.2->-r requirements.txt (line 3)) (0.22.2)
Requirement already satisfied: safetensors>=0.3.1 in /home/chop/.local/lib/python3.10/site-packages (from accelerate==0.27.2->-r requirements.txt (line 3)) (0.4.3)
Requirement already satisfied: tqdm in /home/chop/.local/lib/python3.10/site-packages (from peft==0.8.2->-r requirements.txt (line 12)) (4.66.2)
Requirement already satisfied: regex!=2019.12.17 in /home/chop/.local/lib/python3.10/site-packages (from transformers==4.39.0->-r requirements.txt (line 13)) (2024.4.16)
Requirement already satisfied: requests in /home/chop/.local/lib/python3.10/site-packages (from transformers==4.39.0->-r requirements.txt (line 13)) (2.31.0)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /home/chop/.local/lib/python3.10/site-packages (from transformers==4.39.0->-r requirements.txt (line 13)) (0.15.2)
Collecting hjson (from deepspeed==0.14.0->-r requirements.txt (line 14))Using cached https://pypi.tuna.tsinghua.edu.cn/packages/1f/7f/13cd798d180af4bf4c0ceddeefba2b864a63c71645abc0308b768d67bb81/hjson-3.1.0-py3-none-any.whl (54 kB)
Collecting ninja (from deepspeed==0.14.0->-r requirements.txt (line 14))Using cached https://pypi.tuna.tsinghua.edu.cn/packages/6d/92/8d7aebd4430ab5ff65df2bfee6d5745f95c004284db2d8ca76dcbfd9de47/ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (307 kB)
Collecting py-cpuinfo (from deepspeed==0.14.0->-r requirements.txt (line 14))Using cached https://pypi.tuna.tsinghua.edu.cn/packages/e0/a9/023730ba63db1e494a271cb018dcd361bd2c917ba7004c3e49d5daf795a2/py_cpuinfo-9.0.0-py3-none-any.whl (22 kB)
Requirement already satisfied: pydantic in /home/chop/.local/lib/python3.10/site-packages (from deepspeed==0.14.0->-r requirements.txt (line 14)) (2.7.0)
Collecting pynvml (from deepspeed==0.14.0->-r requirements.txt (line 14))Using cached https://pypi.tuna.tsinghua.edu.cn/packages/5b/9c/adb8070059caaa15d5a572b66bccd95900d8c1b9fa54d6ecea6ae97448d1/pynvml-11.5.0-py3-none-any.whl (53 kB)
Requirement already satisfied: nvidia-nvjitlink-cu12 in /home/chop/.local/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch==2.1.2->-r requirements.txt (line 1)) (12.4.127)
Requirement already satisfied: python-dateutil>=2.8.2 in /home/chop/.local/lib/python3.10/site-packages (from pandas->-r requirements.txt (line 6)) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /home/chop/.local/lib/python3.10/site-packages (from pandas->-r requirements.txt (line 6)) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /home/chop/.local/lib/python3.10/site-packages (from pandas->-r requirements.txt (line 6)) (2024.1)
Requirement already satisfied: pyarrow>=12.0.0 in /home/chop/.local/lib/python3.10/site-packages (from datasets->-r requirements.txt (line 9)) (16.0.0)
Requirement already satisfied: pyarrow-hotfix in /home/chop/.local/lib/python3.10/site-packages (from datasets->-r requirements.txt (line 9)) (0.6)
Requirement already satisfied: dill<0.3.9,>=0.3.0 in /home/chop/.local/lib/python3.10/site-packages (from datasets->-r requirements.txt (line 9)) (0.3.8)
Requirement already satisfied: xxhash in /home/chop/.local/lib/python3.10/site-packages (from datasets->-r requirements.txt (line 9)) (3.4.1)
Requirement already satisfied: multiprocess in /home/chop/.local/lib/python3.10/site-packages (from datasets->-r requirements.txt (line 9)) (0.70.16)
Requirement already satisfied: aiohttp in /home/chop/.local/lib/python3.10/site-packages (from datasets->-r requirements.txt (line 9)) (3.9.5)
Requirement already satisfied: responses<0.19 in /home/chop/.local/lib/python3.10/site-packages (from evaluate->-r requirements.txt (line 10)) (0.18.0)
Requirement already satisfied: iniconfig in /home/chop/.local/lib/python3.10/site-packages (from pytest->-r requirements.txt (line 11)) (2.0.0)
Requirement already satisfied: pluggy<2.0,>=1.4 in /home/chop/.local/lib/python3.10/site-packages (from pytest->-r requirements.txt (line 11)) (1.5.0)
Requirement already satisfied: exceptiongroup>=1.0.0rc8 in /home/chop/.local/lib/python3.10/site-packages (from pytest->-r requirements.txt (line 11)) (1.2.1)
Requirement already satisfied: tomli>=1 in /home/chop/.local/lib/python3.10/site-packages (from pytest->-r requirements.txt (line 11)) (2.0.1)
Requirement already satisfied: joblib>=1.2.0 in /home/chop/.local/lib/python3.10/site-packages (from scikit-learn->-r requirements.txt (line 15)) (1.4.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in /home/chop/.local/lib/python3.10/site-packages (from scikit-learn->-r requirements.txt (line 15)) (3.4.0)
INFO: pip is looking at multiple versions of torchvision to determine which version is compatible with other requirements. This could take a while.
Collecting torchvision (from -r requirements.txt (line 16))Using cached https://pypi.tuna.tsinghua.edu.cn/packages/1a/e8/13432ae6be567b577a4c89d1bd50084e4d989b379a7be8050380b5ab3a6e/torchvision-0.17.1-cp310-cp310-manylinux1_x86_64.whl (6.9 MB)Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d8/51/55393d57c2d95311b1675c8cd37d307f5022460cf98746e4df882dfb415c/torchvision-0.17.0-cp310-cp310-manylinux1_x86_64.whl (6.9 MB)Using cached https://pypi.tuna.tsinghua.edu.cn/packages/da/ae/76bd3682465730dea7be21f36a8160a911a470de6f26228904f222e7fefe/torchvision-0.16.2-cp310-cp310-manylinux1_x86_64.whl (6.8 MB)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /home/chop/.local/lib/python3.10/site-packages (from torchvision->-r requirements.txt (line 16)) (10.3.0)
Requirement already satisfied: urllib3>=1.25 in /home/chop/.local/lib/python3.10/site-packages (from torchdata->-r requirements.txt (line 17)) (2.2.1)
INFO: pip is looking at multiple versions of torchaudio to determine which version is compatible with other requirements. This could take a while.
Collecting torchaudio (from -r requirements.txt (line 18))Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ce/99/7485966a902905e206eda57fdca8de69545c107e33eefe9f6536c2dda16c/torchaudio-2.2.1-cp310-cp310-manylinux1_x86_64.whl (3.3 MB)Using cached https://pypi.tuna.tsinghua.edu.cn/packages/30/fc/cdcf7c2071539ea147ddb6de2b538d9c1599665b621f2e6cf0b3ef51d20d/torchaudio-2.2.0-cp310-cp310-manylinux1_x86_64.whl (3.3 MB)Using cached https://pypi.tuna.tsinghua.edu.cn/packages/25/83/4170df23c16c25818ce9591fd2b8109cda3f725d115d77417d4efef4eb46/torchaudio-2.1.2-cp310-cp310-manylinux1_x86_64.whl (3.3 MB)
Requirement already satisfied: absl-py>=0.4 in /home/chop/.local/lib/python3.10/site-packages (from tensorboard->-r requirements.txt (line 19)) (2.1.0)
Requirement already satisfied: grpcio>=1.48.2 in /home/chop/.local/lib/python3.10/site-packages (from tensorboard->-r requirements.txt (line 19)) (1.62.2)
Requirement already satisfied: markdown>=2.6.8 in /home/chop/.local/lib/python3.10/site-packages (from tensorboard->-r requirements.txt (line 19)) (3.6)
Requirement already satisfied: protobuf!=4.24.0,>=3.19.6 in /home/chop/.local/lib/python3.10/site-packages (from tensorboard->-r requirements.txt (line 19)) (5.26.1)
Requirement already satisfied: setuptools>=41.0.0 in /home/chop/miniconda3/envs/Llama-Chinese/lib/python3.10/site-packages (from tensorboard->-r requirements.txt (line 19)) (68.2.2)
Collecting six>1.9 (from tensorboard->-r requirements.txt (line 19))Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl (11 kB)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /home/chop/.local/lib/python3.10/site-packages (from tensorboard->-r requirements.txt (line 19)) (0.7.2)
Requirement already satisfied: werkzeug>=1.0.1 in /home/chop/.local/lib/python3.10/site-packages (from tensorboard->-r requirements.txt (line 19)) (3.0.2)
Requirement already satisfied: aiofiles<24.0,>=22.0 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (23.2.1)
Requirement already satisfied: altair<6.0,>=4.2.0 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (5.3.0)
Requirement already satisfied: fastapi in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (0.110.2)
Requirement already satisfied: ffmpy in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (0.3.2)
Requirement already satisfied: gradio-client==0.15.1 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (0.15.1)
Requirement already satisfied: httpx>=0.24.1 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (0.27.0)
Requirement already satisfied: importlib-resources<7.0,>=1.3 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (6.4.0)
Requirement already satisfied: markupsafe~=2.0 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (2.1.5)
Requirement already satisfied: matplotlib~=3.0 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (3.8.4)
Requirement already satisfied: orjson~=3.0 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (3.10.1)
Requirement already satisfied: pydub in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (0.25.1)
Requirement already satisfied: python-multipart>=0.0.9 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (0.0.9)
Requirement already satisfied: ruff>=0.2.2 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (0.4.1)
Requirement already satisfied: semantic-version~=2.0 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (2.10.0)
Requirement already satisfied: tomlkit==0.12.0 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (0.12.0)
Requirement already satisfied: typer<1.0,>=0.12 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (0.12.3)
Requirement already satisfied: uvicorn>=0.14.0 in /home/chop/.local/lib/python3.10/site-packages (from gradio->-r requirements.txt (line 20)) (0.29.0)
Requirement already satisfied: websockets<12.0,>=10.0 in /home/chop/.local/lib/python3.10/site-packages (from gradio-client==0.15.1->gradio->-r requirements.txt (line 20)) (11.0.3)
Requirement already satisfied: jsonschema>=3.0 in /home/chop/.local/lib/python3.10/site-packages (from altair<6.0,>=4.2.0->gradio->-r requirements.txt (line 20)) (4.21.1)
Requirement already satisfied: toolz in /home/chop/.local/lib/python3.10/site-packages (from altair<6.0,>=4.2.0->gradio->-r requirements.txt (line 20)) (0.12.1)
Requirement already satisfied: aiosignal>=1.1.2 in /home/chop/.local/lib/python3.10/site-packages (from aiohttp->datasets->-r requirements.txt (line 9)) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /home/chop/.local/lib/python3.10/site-packages (from aiohttp->datasets->-r requirements.txt (line 9)) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /home/chop/.local/lib/python3.10/site-packages (from aiohttp->datasets->-r requirements.txt (line 9)) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/chop/.local/lib/python3.10/site-packages (from aiohttp->datasets->-r requirements.txt (line 9)) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /home/chop/.local/lib/python3.10/site-packages (from aiohttp->datasets->-r requirements.txt (line 9)) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /home/chop/.local/lib/python3.10/site-packages (from aiohttp->datasets->-r requirements.txt (line 9)) (4.0.3)
Requirement already satisfied: anyio in /home/chop/.local/lib/python3.10/site-packages (from httpx>=0.24.1->gradio->-r requirements.txt (line 20)) (4.3.0)
Requirement already satisfied: certifi in /home/chop/.local/lib/python3.10/site-packages (from httpx>=0.24.1->gradio->-r requirements.txt (line 20)) (2024.2.2)
Requirement already satisfied: httpcore==1.* in /home/chop/.local/lib/python3.10/site-packages (from httpx>=0.24.1->gradio->-r requirements.txt (line 20)) (1.0.5)
Requirement already satisfied: idna in /home/chop/.local/lib/python3.10/site-packages (from httpx>=0.24.1->gradio->-r requirements.txt (line 20)) (3.7)
Requirement already satisfied: sniffio in /home/chop/.local/lib/python3.10/site-packages (from httpx>=0.24.1->gradio->-r requirements.txt (line 20)) (1.3.1)
Requirement already satisfied: h11<0.15,>=0.13 in /home/chop/.local/lib/python3.10/site-packages (from httpcore==1.*->httpx>=0.24.1->gradio->-r requirements.txt (line 20)) (0.14.0)
Requirement already satisfied: contourpy>=1.0.1 in /home/chop/.local/lib/python3.10/site-packages (from matplotlib~=3.0->gradio->-r requirements.txt (line 20)) (1.2.1)
Requirement already satisfied: cycler>=0.10 in /home/chop/.local/lib/python3.10/site-packages (from matplotlib~=3.0->gradio->-r requirements.txt (line 20)) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/chop/.local/lib/python3.10/site-packages (from matplotlib~=3.0->gradio->-r requirements.txt (line 20)) (4.51.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/chop/.local/lib/python3.10/site-packages (from matplotlib~=3.0->gradio->-r requirements.txt (line 20)) (1.4.5)
Collecting pyparsing>=2.3.1 (from matplotlib~=3.0->gradio->-r requirements.txt (line 20))Using cached https://pypi.tuna.tsinghua.edu.cn/packages/9d/ea/6d76df31432a0e6fdf81681a895f009a4bb47b3c39036db3e1b528191d52/pyparsing-3.1.2-py3-none-any.whl (103 kB)
Requirement already satisfied: annotated-types>=0.4.0 in /home/chop/.local/lib/python3.10/site-packages (from pydantic->deepspeed==0.14.0->-r requirements.txt (line 14)) (0.6.0)
Requirement already satisfied: pydantic-core==2.18.1 in /home/chop/.local/lib/python3.10/site-packages (from pydantic->deepspeed==0.14.0->-r requirements.txt (line 14)) (2.18.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/chop/.local/lib/python3.10/site-packages (from requests->transformers==4.39.0->-r requirements.txt (line 13)) (3.3.2)
Requirement already satisfied: click>=8.0.0 in /home/chop/.local/lib/python3.10/site-packages (from typer<1.0,>=0.12->gradio->-r requirements.txt (line 20)) (8.1.7)
Requirement already satisfied: shellingham>=1.3.0 in /home/chop/.local/lib/python3.10/site-packages (from typer<1.0,>=0.12->gradio->-r requirements.txt (line 20)) (1.5.4)
Requirement already satisfied: rich>=10.11.0 in /home/chop/.local/lib/python3.10/site-packages (from typer<1.0,>=0.12->gradio->-r requirements.txt (line 20)) (13.7.1)
Requirement already satisfied: starlette<0.38.0,>=0.37.2 in /home/chop/.local/lib/python3.10/site-packages (from fastapi->gradio->-r requirements.txt (line 20)) (0.37.2)
Requirement already satisfied: mpmath>=0.19 in /home/chop/.local/lib/python3.10/site-packages (from sympy->torch==2.1.2->-r requirements.txt (line 1)) (1.3.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/chop/.local/lib/python3.10/site-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio->-r requirements.txt (line 20)) (2023.12.1)
Requirement already satisfied: referencing>=0.28.4 in /home/chop/.local/lib/python3.10/site-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio->-r requirements.txt (line 20)) (0.34.0)
Requirement already satisfied: rpds-py>=0.7.1 in /home/chop/.local/lib/python3.10/site-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio->-r requirements.txt (line 20)) (0.18.0)
Requirement already satisfied: markdown-it-py>=2.2.0 in /home/chop/.local/lib/python3.10/site-packages (from rich>=10.11.0->typer<1.0,>=0.12->gradio->-r requirements.txt (line 20)) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /home/chop/.local/lib/python3.10/site-packages (from rich>=10.11.0->typer<1.0,>=0.12->gradio->-r requirements.txt (line 20)) (2.17.2)
Requirement already satisfied: mdurl~=0.1 in /home/chop/.local/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich>=10.11.0->typer<1.0,>=0.12->gradio->-r requirements.txt (line 20)) (0.1.2)
Installing collected packages: py-cpuinfo, ninja, hjson, triton, six, pyyaml, pyparsing, pynvml, nvidia-nccl-cu12, torch, torchvision, torchaudio, deepspeedAttempting uninstall: tritonFound existing installation: triton 2.2.0Uninstalling triton-2.2.0:Successfully uninstalled triton-2.2.0Attempting uninstall: nvidia-nccl-cu12Found existing installation: nvidia-nccl-cu12 2.19.3Uninstalling nvidia-nccl-cu12-2.19.3:Successfully uninstalled nvidia-nccl-cu12-2.19.3Attempting uninstall: torchFound existing installation: torch 2.2.2Uninstalling torch-2.2.2:Successfully uninstalled torch-2.2.2Attempting uninstall: torchvisionFound existing installation: torchvision 0.17.2Uninstalling torchvision-0.17.2:Successfully uninstalled torchvision-0.17.2Attempting uninstall: torchaudioFound existing installation: torchaudio 2.2.2Uninstalling torchaudio-2.2.2:Successfully uninstalled torchaudio-2.2.2
Successfully installed deepspeed-0.14.0 hjson-3.1.0 ninja-1.11.1.1 nvidia-nccl-cu12-2.18.1 py-cpuinfo-9.0.0 pynvml-11.5.0 pyparsing-3.1.2 pyyaml-6.0.1 six-1.16.0 torch-2.1.2 torchaudio-2.1.2 torchvision-0.16.2 triton-2.1.0
(Llama-Chinese) :~/Llama-Chinese/Llama-Chinese$ pip list
Package Version
------------------------- -----------
absl-py 2.1.0
accelerate 0.27.2
aiofiles 23.2.1
aiohttp 3.9.5
aiosignal 1.3.1
altair 5.3.0
annotated-types 0.6.0
anyio 4.3.0
async-timeout 4.0.3
attrs 23.2.0
bitsandbytes 0.42.0
certifi 2024.2.2
charset-normalizer 3.3.2
click 8.1.7
contourpy 1.2.1
cycler 0.12.1
datasets 2.19.0
deepspeed 0.14.0
dill 0.3.8
evaluate 0.4.1
exceptiongroup 1.2.1
fastapi 0.110.2
ffmpy 0.3.2
filelock 3.13.4
fonttools 4.51.0
frozenlist 1.4.1
fsspec 2024.3.1
gekko 1.0.6
gradio 4.27.0
gradio_client 0.15.1
grpcio 1.62.2
h11 0.14.0
hjson 3.1.0
httpcore 1.0.5
httpx 0.27.0
huggingface-hub 0.22.2
idna 3.7
importlib_resources 6.4.0
iniconfig 2.0.0
Jinja2 3.1.3
joblib 1.4.0
jsonschema 4.21.1
jsonschema-specifications 2023.12.1
kiwisolver 1.4.5
Markdown 3.6
markdown-it-py 3.0.0
MarkupSafe 2.1.5
matplotlib 3.8.4
mdurl 0.1.2
mpmath 1.3.0
multidict 6.0.5
multiprocess 0.70.16
networkx 3.3
ninja 1.11.1.1
numpy 1.26.4
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.18.1
nvidia-nvjitlink-cu12 12.4.127
nvidia-nvtx-cu12 12.1.105
orjson 3.10.1
packaging 24.0
pandas 2.2.2
peft 0.8.2
pillow 10.3.0
pip 23.3.1
pluggy 1.5.0
protobuf 5.26.1
psutil 5.9.8
py-cpuinfo 9.0.0
pyarrow 16.0.0
pyarrow-hotfix 0.6
pydantic 2.7.0
pydantic_core 2.18.1
pydub 0.25.1
Pygments 2.17.2
pynvml 11.5.0
pyparsing 3.1.2
pytest 8.1.1
python-dateutil 2.9.0.post0
python-multipart 0.0.9
pytz 2024.1
PyYAML 6.0.1
referencing 0.34.0
regex 2024.4.16
requests 2.31.0
responses 0.18.0
rich 13.7.1
rpds-py 0.18.0
ruff 0.4.1
safetensors 0.4.3
scikit-learn 1.4.2
scipy 1.13.0
semantic-version 2.10.0
sentencepiece 0.2.0
setuptools 68.2.2
shellingham 1.5.4
six 1.16.0
sniffio 1.3.1
starlette 0.37.2
sympy 1.12
tensorboard 2.16.2
tensorboard-data-server 0.7.2
threadpoolctl 3.4.0
tokenizers 0.15.2
tomli 2.0.1
tomlkit 0.12.0
toolz 0.12.1
torch 2.1.2
torchaudio 2.1.2
torchdata 0.7.1
torchvision 0.16.2
tqdm 4.66.2
transformers 4.39.0
triton 2.1.0
typer 0.12.3
typing_extensions 4.11.0
tzdata 2024.1
urllib3 2.2.1
uvicorn 0.29.0
websockets 11.0.3
Werkzeug 3.0.2
wheel 0.41.2
xxhash 3.4.1
yarl 1.9.4下载模型：Atom-7B-Chat
你可以从以下来源下载Atom-7B-Chat模型。
https://github.com/LlamaFamily/Llama-Chinese?tab=readme-ov-file
中文预训练模型Atom
原子大模型Atom由Llama中文社区和原子回声联合打造。
| 类别 | 模型名称 | 🤗模型加载名称 | 下载地址 |
|---|---|---|---|
| 预训练 | Atom-7B | FlagAlpha/Atom-7B | HuggingFace | ModelScope | WiseModel |
| Chat | Atom-7B-Chat | FlagAlpha/Atom-7B-Chat | HuggingFace | ModelScope | WiseModel |
Atom系列模型包含Atom-13B、Atom-7B和Atom-1B，基于Llama2做了中文能力的持续优化。Atom-7B和Atom-7B-Chat目前已完全开源，支持商用，可在Hugging Face仓库获取模型，详情见Atom-7B下载。Atom大模型针对中文做了以下优化：
- 大规模的中文数据预训练
原子大模型Atom在Llama2的基础上，采用大规模的中文数据进行持续预训练，包含百科、书籍、博客、新闻、公告、小说、金融数据、法律数据、医疗数据、代码数据、专业论文数据、中文自然语言处理竞赛数据集等，详见📝 数据来源。
同时对庞大的数据进行了过滤、打分、去重，筛选出超过1T token的高质量中文数据，持续不断加入训练迭代中。
- 更高效的中文词表 为了提高中文文本处理的效率，我们针对Llama2模型的词表进行了深度优化。首先，我们基于数百G的中文文本，在该模型词表的基础上扩展词库至65,000个单词。经过测试，我们的改进使得中文编码/解码速度提高了约350％。此外，我们还扩大了中文字符集的覆盖范围，包括所有emoji符号😊。这使得生成带有表情符号的文章更加高效。
- 自适应上下文扩展 Atom大模型默认支持4K上下文，利用位置插值PI和Neural Tangent Kernel （NTK）方法，经过微调可以将上下文长度扩增到32K。
- 📝 中文数据
我们通过以下数据来优化Llama2的中文能力:
| 类型 | 描述 |
|---|---|
| 网络数据 | 互联网上公开的网络数据，挑选出去重后的高质量中文数据，涉及到百科、书籍、博客、新闻、公告、小说等高质量长文本数据。 |
| Wikipedia | 中文Wikipedia的数据 |
| 悟道 | 中文悟道开源的200G数据 |
| Clue | Clue开放的中文预训练数据，进行清洗后的高质量中文长文本数据 |
| 竞赛数据集 | 近年来中文自然语言处理多任务竞赛数据集，约150个 |
| MNBVC | MNBVC 中清洗出来的部分数据集 |
社区提供预训练版本Atom-7B和基于Atom-7B进行对话微调的模型参数供开放下载，关于模型的进展详见社区官网llama.family。
(Llama-Chinese) :~$ cd Atom-7B-Chat
(Llama-Chinese) :~/Atom-7B-Chat$ ls
README.md generation_config.json model-00002-of-00003.safetensors model_atom.py tokenizer_config.json
config.json gitattributes model-00003-of-00003.safetensors special_tokens_map.json
configuration_atom.py model-00001-of-00003.safetensors model.safetensors.index.json tokenizer.model模型推理
使用Atom-7B-Chat模型进行推理 创建一个名为 quick_start.py 的文件，并将以下内容复制到该文件中。
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
device_map = "cuda:0" if torch.cuda.is_available() else "auto"
model = AutoModelForCausalLM.from_pretrained('./Atom-7B-Chat',device_map=device_map,torch_dtype=torch.float16,load_in_8bit=True,trust_remote_code=True,use_fla
sh_attention_2=False)
model =model.eval()
tokenizer = AutoTokenizer.from_pretrained('./Atom-7B-Chat',use_fast=False)
tokenizer.pad_token = tokenizer.eos_token
input_ids = tokenizer(['<s>Human: 介绍一下中国\n</s><s>Assistant: '], return_tensors="pt",add_special_tokens=False).input_ids
if torch.cuda.is_available():input_ids = input_ids.to('cuda')
generate_input = {"input_ids":input_ids,"max_new_tokens":512,"do_sample":True,"top_k":50,"top_p":0.95,"temperature":0.3,"repetition_penalty":1.3,"eos_token_id":tokenizer.eos_token_id,"bos_token_id":tokenizer.bos_token_id,"pad_token_id":tokenizer.pad_token_id
}
generate_ids = model.generate(**generate_input)
text = tokenizer.decode(generate_ids[0])
print(text)执行脚本
(Llama-Chinese) :~/Atom-7B-Chat$ python quick_start.py
The `load_in_4bit` and `load_in_8bit` arguments are deprecated and will be removed in the future versions. Please, pass a `BitsAndBytesConfig` object in `quantization_config` argument instead.
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:11<00:00, 3.97s/it]
<s> Human: 介绍一下中国
</s><s> Assistant: 中华人民共和国简称“中”，位于亚洲东部、太平洋西岸。领土总面积960万平方公里（含领海），人口14亿多(2021年)居世界第一；是世界上历史悠久的国家之一，有着5000多年的文明史，也是世界上最早发明文字的四大古国之一。自公元前7世纪起开始出现统一的国家政权——秦朝后历经多个朝代更替直到新中国成立至今一直保持统一状态并成为当今世界的重要大国之一。
</s>
(Llama-Chinese) :~$nvidia-smi命令实时查看指定GPU使用情况
watch -n 1 nvidia-smi # 1代表每隔1秒刷新一次GPU使用情况NVIDIA-SMI 550.76.01 #GRID版本
Driver Version: 552.22 #驱动版本
CUDA Version: 12.4 #CUDA最高支持的版本
GPU：本机中的GPU编号，从0开始，本机只有一块GPU
Fan：风扇转速（0%-100%），N/A表示没有风扇
Name：GPU名字/类型，NVIDIA GeForce RTX 3080TI
Temp：GPU温度（GPU温度过高会导致GPU频率下降） 66C
Perf：性能状态，从P0（最大性能）到P12（最小性能），显示P0，最大性能
Pwr：Usager/Cap：GPU功耗，Usage表示用了多少，Cap表示总共多少 ， 43W / 80W
Persistence-M：持续模式状态，持续模式耗能大，为On
Bus-Id：GPU总线 00000000:01:00.0
Disp.A：Display Active，表示GPU是否初始化 Off
Memory-Usage：显存使用率 7746MiB / 16384MiB，表示已接近占满
Volatile GPU-UTil：GPU使用率，32%
Uncorr. ECC：是否开启错误检查和纠错技术，0/DISABLED，1/ENABLED，为N/A
Compute M：计算模式，0/DEFAULT，1/EXCLUSIVE_PROCESS，2/PROHIBITED，为Default
Processes：显示每个进程占用的显存使用率、进程号、占用的哪个GPU，/python3.10
GPU Memory Usage #该进程占用的显存。
附：如果不安装flash_attn会报错。使用pip install flash_attn安装，在脚本里设置：use_flash_attention_2=False
(Llama-Chinese) :~/Atom-7B-Chat$ python quick_start.py
Traceback (most recent call last):File "/home/chop/Atom-7B-Chat/quick_start.py", line 4, in <module>model = AutoModelForCausalLM.from_pretrained('./Atom-7B-Chat',device_map=device_map,torch_dtype=torch.float16,load_in_8bit=True,trust_remote_code=True,use_flash_attention_2=False)File "/home/chop/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 550, in from_pretrainedmodel_class = get_class_from_dynamic_module(File "/home/chop/.local/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 489, in get_class_from_dynamic_modulefinal_module = get_cached_module_file(File "/home/chop/.local/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 315, in get_cached_module_filemodules_needed = check_imports(resolved_module_file)File "/home/chop/.local/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 180, in check_importsraise ImportError(
ImportError: This modeling file requires the following packages that were not found in your environment: flash_attn. Run `pip install flash_attn`
(Llama-Chinese) :~/Atom-7B-Chat$ pip install flash_attn
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting flash_attnUsing cached flash_attn-2.5.7-cp310-cp310-linux_x86_64.whl
Requirement already satisfied: torch in /home/chop/miniconda3/envs/Llama-Chinese/lib/python3.10/site-packages (from flash_attn) (2.1.2)
Requirement already satisfied: einops in /home/chop/miniconda3/envs/Llama-Chinese/lib/python3.10/site-packages (from flash_attn) (0.7.0)
Requirement already satisfied: packaging in /home/chop/.local/lib/python3.10/site-packages (from flash_attn) (24.0)
Requirement already satisfied: ninja in /home/chop/miniconda3/envs/Llama-Chinese/lib/python3.10/site-packages (from flash_attn) (1.11.1.1)
Requirement already satisfied: filelock in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (3.13.4)
Requirement already satisfied: typing-extensions in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (4.11.0)
Requirement already satisfied: sympy in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (1.12)
Requirement already satisfied: networkx in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (3.3)
Requirement already satisfied: jinja2 in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (3.1.3)
Requirement already satisfied: fsspec in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (2024.3.1)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (12.1.105)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (12.1.105)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (12.1.105)
Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (8.9.2.26)
Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (12.1.3.1)
Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (11.0.2.54)
Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (10.3.2.106)
Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (11.4.5.107)
Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (12.1.0.106)
Requirement already satisfied: nvidia-nccl-cu12==2.18.1 in /home/chop/miniconda3/envs/Llama-Chinese/lib/python3.10/site-packages (from torch->flash_attn) (2.18.1)
Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /home/chop/.local/lib/python3.10/site-packages (from torch->flash_attn) (12.1.105)
Requirement already satisfied: triton==2.1.0 in /home/chop/miniconda3/envs/Llama-Chinese/lib/python3.10/site-packages (from torch->flash_attn) (2.1.0)
Requirement already satisfied: nvidia-nvjitlink-cu12 in /home/chop/.local/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch->flash_attn) (12.4.127)
Requirement already satisfied: MarkupSafe>=2.0 in /home/chop/.local/lib/python3.10/site-packages (from jinja2->torch->flash_attn) (2.1.5)
Requirement already satisfied: mpmath>=0.19 in /home/chop/.local/lib/python3.10/site-packages (from sympy->torch->flash_attn) (1.3.0)
Installing collected packages: flash_attn
Successfully installed flash_attn-2.5.7e-packages (from torch->flash_attn) (2.1.0)
Requirement already satisfied: nvidia-nvjitlink-cu12 in /home/chop/.local/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch->flash_attn) (12.4.127)
Requirement already satisfied: MarkupSafe>=2.0 in /home/chop/.local/lib/python3.10/site-packages (from jinja2->torch->flash_attn) (2.1.5)
Requirement already satisfied: mpmath>=0.19 in /home/chop/.local/lib/python3.10/site-packages (from sympy->torch->flash_attn) (1.3.0)
Installing collected packages: flash_attn
Successfully installed flash_attn-2.5.7
