K2Horizon llama.cpp CUDA Binary
Prebuilt Linux x86_64 CUDA binaries for the model/K2Horizon branch of MBZUAI-IFM llama.cpp.
This package is already built. You do not need to compile llama.cpp again.
Build Information
- Repository: https://github.com/MBZUAI-IFM/llama.cpp
- Branch: model/K2Horizon
- Platform: Linux x86_64
- CUDA: Enabled
- Version: 0.3.0-dev
- Build: 10671
- Commit: 35999d101
Important
The package includes its own shared libraries inside bin/.
Before running llama-server or llama-cli, set:
export LD_LIBRARY_PATH="$PWD/bin:$LD_LIBRARY_PATH"
You can also add the binaries to PATH:
export PATH="$PWD/bin:$PATH"
Quick Start
Extract the archive:
tar -xzf llama-k2horizon-cuda-linux-x64.tar.gz
Enter the directory:
cd llama-k2horizon-cuda-linux-x64
Set the runtime library path:
export LD_LIBRARY_PATH="$PWD/bin:$LD_LIBRARY_PATH"
Optional PATH:
export PATH="$PWD/bin:$PATH"
Check the binary:
llama-server --version
Check CUDA devices:
llama-server --list-devices
Run a GGUF Model
llama-server -m /path/to/model.gguf -ngl 999
Run CLI
llama-cli -m /path/to/model.gguf -ngl 999 -p "Hello"
Included
The bin directory contains llama-server, llama-cli, CUDA GGML libraries, CPU GGML libraries, libllama shared libraries, and other utilities from the existing build.
Requirements
- Linux x86_64
- NVIDIA GPU
- NVIDIA driver
- Compatible CUDA driver support
Check GPU availability with:
nvidia-smi
Notes
This package contains prebuilt binaries and runtime libraries. The source code and CMake build directory are not required for normal inference.