chatglm

go-chatglm.cpp

chatglm.cpp golang bindings.

The go-chatglm.cpp bindings are high level, as such most of the work is kept into the C/C++ code to avoid any extra computational cost, be more performant and lastly ease out maintenance, while keeping the usage as simple as possible.

Attention！

Environment

You need to make sure there are make, cmake, gcc command in your machine, otherwise should support C++17.

If you want to run on Windows OS, you can use cygwin or MinGW.

cmake > 3.8 and gcc > 5.1.0 (support C++17)

Not Support LoRA model

go-chatglm.cpp is not anymore compatible with LoRA model, but it woks ONLY with the model which merged by LoRA model and base model.

You can use convert.py in chatglm.cpp to merge LoRA model into base model.

Usage

Note: This repository uses git submodules to keep track of chatglm.cpp .

Clone the repository locally:

git clone --recurse-submodules https://github.com/Weaxs/go-chatglm.cpp

To build the bindings locally, run:

cd go-chatglm.cpp
make libbinding.a

Now you can run the example with:

go run ./examples -m "/model/path/here"
                    ____ _           _    ____ _     __  __                   
  __ _  ___        / ___| |__   __ _| |_ / ___| |   |  \/  |  ___ _ __  _ __  
 / _` |/ _ \ _____| |   | '_ \ / _` | __| |  _| |   | |\/| | / __| '_ \| '_ \ 
| (_| | (_) |_____| |___| | | | (_| | |_| |_| | |___| |  | || (__| |_) | |_) |
 \__, |\___/       \____|_| |_|\__,_|\__|\____|_____|_|  |_(_)___| .__/| .__/ 
 |___/                                                           |_|   |_|    

>>> 你好

Sending 你好


你好👋！我是人工智能助手 ChatGLM3-6B，很高兴见到你，欢迎问我任何问题。

Acceleration

Metal (Apple Silicon)

MPS (Metal Performance Shaders) allows computation to run on powerful Apple Silicon GPU.

BUILD_TYPE=metal make libbinding.a
go build -tags metal ./examples/main.go
./main -m "/model/path/here"

OpenBLAS

OpenBLAS provides acceleration on CPU.

BUILD_TYPE=openblas make libbinding.a
go build -tags openblas ./examples/main.go
./main -m "/model/path/here"

cuBLAS

cuBLAS uses NVIDIA GPU to accelerate BLAS.

BUILD_TYPE=cublas make libbinding.a
go build -tags cublas ./examples/main.go
./main -m "/model/path/here"

Acknowledgements

• This project is greatly inspired by @mudler's go-llama.cpp