| 其他封装 | |
|---|
| Tensor library for machine learning, inspired by ggml | https://github.com/candrewlee14/zgml |
| Bleeding edge low level Rust binding for GGML | https://github.com/KerfuffleV2/ggml-sys-bleedingedge |
| Your customized AI characters - personal assistants on any hardware! With llama.cpp, whisper.cpp, ggml, LLaMA-v2. | https://github.com/nrl-ai/CustomChar |
| cpu inference example for mpt-7b with ggml+rust | https://github.com/apage43/mptgen |
| Python bindings for the GGML GPT-J Laguage model | https://github.com/abdeladim-s/pygptj |
| Run any ggml ported ML models directly on your web browser with ggml.js | https://github.com/rahuldshetty/ggml.js |
| Python bindings for the Transformer models implemented in C/C++ using GGML library. | https://github.com/marella/ctransformers |
ggml-python
- 在github上有一个使用python对ggml进行封装的项目ggml-python，通过下面的例子可以看出使用ggml进行推理主要包括以下几个步骤：
- 上下文环境创建=>
- tensors数据初始化=>
- 构建计算图=>
- 设置tensor值=>
- 前向推理=>
- 输出值，释放上下文<=>
# https://github.com/abetlen/ggml-python
import ggml
import ctypes# Allocate a new context with 16 MB of memory
params = ggml.ggml_init_params(mem_size=16 * 1024 * 1024, mem_buffer=None)
ctx = ggml.ggml_init(params=params)# Instantiate tensors
x = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
a = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
b = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)# Use ggml operations to build a computational graph
x2 = ggml.ggml_mul(ctx, x, x)
f = ggml.ggml_add(ctx, ggml.ggml_mul(ctx, a, x2), b)gf = ggml.ggml_build_forward(f)# Set the input values
ggml.ggml_set_f32(x, 2.0)
ggml.ggml_set_f32(a, 3.0)
ggml.ggml_set_f32(b, 4.0)# Compute the graph
ggml.ggml_graph_compute_with_ctx(ctx, ctypes.pointer(gf), 1)# Get the output value
output = ggml.ggml_get_f32_1d(f, 0)
assert output == 16.0# Free the context
ggml.ggml_free(ctx)
CG
- https://ggml-python.readthedocs.io/en/latest/
- https://github.com/ggerganov/ggml/tree/master/examples/python
- https://github.com/ggerganov/whisper.cpp/issues/9
- https://github.com/go-skynet/LocalAI
