Teknosys learning library

Compile GGUF tools on Fedora with llama.cpp

To run a GGUF model on Fedora, compile an inference runtime such as llama.cpp. GGUF is a binary model file format used by GGML-based tools, not a library called “Generalized Gaussian Unsupervised Feature.” A model file is loaded by a program; it is not compiled by CMake. This guide builds the programs that read it.

1. Install the build tools

On a conventional Fedora installation, install a C/C++ compiler, Git, CMake, and the development headers for the runtime's HTTPS support. These commands install packages on your machine; review them before running.

sudo dnf install git gcc gcc-c++ cmake make openssl-devel

Fedora Atomic desktops have a different host-management workflow. Use a development container such as Toolbox and follow the upstream Fedora guidance rather than applying these host package commands unchanged. JPEG, PNG, and LAPACK headers are not required for this basic text-only build.

2. Clone the inference runtime

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
git rev-parse HEAD

Record the commit printed by the last command together with your Fedora release and compiler version. The upstream default branch changes over time; for repeatable deployments, review and check out a specific commit or release before building. Do not use an unrelated repository merely because its name resembles GGUF.

3. Configure and compile a CPU build

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release --parallel 4

Four build jobs are a conservative starting point, not a performance requirement. Increase the count if you have spare CPU and RAM; reduce it if the compiler is killed by the system's out-of-memory handling. The build directory separates generated artifacts from source. No system-wide installation or root build is needed.

4. Verify the executable and run a compatible model

./build/bin/llama-cli --version
./build/bin/llama-cli --help
./build/bin/llama-cli -m /absolute/path/to/model.gguf -p "Explain GGUF briefly." -n 128

Replace the model path with an actual, compatible GGUF file from a source you trust. Building the runtime does not download a model, convert a Hugging Face checkpoint, or train new weights. Check the model license, architecture support, file integrity, and memory requirements first. An arbitrary GGUF file is not necessarily supported by every runtime version.

5. Optional CPU BLAS acceleration

sudo dnf install openblas-devel
cmake -S . -B build-blas -DCMAKE_BUILD_TYPE=Release -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS
cmake --build build-blas --config Release --parallel 4

OpenBLAS remains a CPU backend. It can help some workloads, especially prompt processing, but it does not make an NVIDIA GPU active and does not guarantee faster token generation. Compare the same model and prompt before deciding which build to use.

6. Optional NVIDIA CUDA build

Install a compatible NVIDIA driver and CUDA toolkit using the vendor and upstream Fedora instructions linked below. Package names and supported compiler versions vary by Fedora and CUDA release. Confirm both driver access and toolkit availability; a working nvidia-smi alone does not prove that the CUDA compiler is installed.

nvidia-smi
nvcc --version
cmake -S . -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build-cuda --config Release --parallel 4
./build-cuda/bin/llama-cli -m /absolute/path/to/model.gguf -ngl 99 -p "Explain GPU offloading." -n 128

The GPU-layer setting requests offloading; the actual allocation depends on the model, available VRAM, and runtime configuration. Read the startup logs and watch nvidia-smi during generation to verify what ran on the GPU. Do not assume that both GPUs are used simply because they are visible.

Troubleshooting and next steps

• CMake cannot find OpenSSL: check that openssl-devel is installed in the same environment where you build.
• CUDA compiler or host compiler errors: consult the compatibility requirements for your installed toolkit rather than disabling checks blindly.
• Model load failure: verify the path, file completeness, supported architecture, and available RAM or VRAM.
• No gguf command: this is expected. Verify llama-cli instead; there is no generic gguf --version step in this workflow.

For conversion rather than runtime compilation, continue with Create a GGUF model on Fedora and its companion conversion and quantization articles. Keep the original checkpoint available so you can regenerate the output when tooling changes.

Sources: GGUF format specification; llama.cpp build documentation; Fedora CUDA and Toolbox guidance; NVIDIA CUDA installation guide. Commands are instructions for readers, not evidence of a build tested on every Fedora release.

Educational material. Verify examples against current documentation and applicable licenses. Report a correction.