OSS Tanbou
View on GitHub

llama.cpp — C/C++ LLM inference across GGUF, quantization, CPUs, GPUs, and local APIs

OSS health 93
About these scores

Discovery score is an unbounded weighted, log-compressed index of stars, watchers, forks, and contributors. Growth momentum is its change over the observed period; OSS health is a 0–100 score from available repository recency, Community Health, and release data.

Stars
128,542
Primary language
C++
License
MIT
Repository last updated
Sep 17, 2026

Overview

llama.cpp is a C/C++ inference runtime for GGUF models. It supports low-bit quantization, CPU+GPU hybrid inference, CUDA, Metal, Vulkan, SYCL, and other backends, plus a CLI, OpenAI-compatible API server, and built-in web UI. The latest release checked on September 17, 2026 is v0.4.1.

Features and best fit

Based on official documentation; not hands-on tested · Content checked:

Run LLMs and VLMs across CPUs, NVIDIA and AMD GPUs, Apple Silicon, and more

Backends include x86 AVX variants, Apple Silicon through NEON/Accelerate/Metal, NVIDIA CUDA, AMD HIP, Vulkan, SYCL, OpenCL, and others. CPU+GPU hybrid inference can also split models larger than available VRAM.

Sources: [2]

Use GGUF and low-bit quantization to tune memory and speed, then expose CLI or API workflows

Quantization options from 1.5-bit through 8-bit let users trade model size, memory, and inference performance. llama cli provides local interaction while llama serve exposes an OpenAI-compatible API and web UI.

Sources: [2]

For developers who want local inference tuned to specific hardware

It is a strong fit for desktop, server, and edge deployments where model format, quantization, and compute backend need to be selected explicitly.

Sources: [2]

Performance is hardware- and model-dependent, and the project evolves quickly

Benchmark on the target hardware because architecture, context length, quantization, backend, and memory bandwidth materially affect results. v0.4.1 updated model support, JSON-schema handling, server process management, logging, and deprecated options.

Sources: [3]

Official sources

  1. [1]ggml-org/llama.cpp — GitHub repository metadata(2026-09-17)
  2. [2]llama.cpp — README(2026-09-17)
  3. [3]llama.cpp v0.4.1 release(2026-09-17)
Supplemental curator note

Its value is fine-grained control over model format, quantization, and hardware backends for local AI. Benchmark on the target hardware.

Try it in 3 steps

  1. 1

    Get the source

    git clone --depth 1 https://github.com/ggml-org/llama.cpp.git
  2. 2

    Enter the repository

    cd llama.cpp
  3. 3

    Check the official steps

    Continue with the commands in the README Installation, Quick Start, or Getting Started section.

    find . -maxdepth 1 -iname 'README*' -exec sed -n '1,220p' {} \; -quit
Check the official README

Growth

Growth trends · Last 30 days

128,542 Stars

Trend data is still being collected.

Development activity

Last 90 days · weekly

Commits (last 30 days)
537
Open PRs
1,587

Development activity is still being collected.

Built with

Categories and tags

Categories

Tags

GitHub data

GitHub dataView detailed GitHub data
View on GitHub

GitHub Topics

  • ggml
Stars
128,542
Forks
23,317
Watchers
842
Open issues
888
Primary language
C++
License
MIT
Repository last updated
Sep 17, 2026
Write a related article

Share a guide or use case for this OSS in Markdown. Articles are published after administrator approval.

Report incorrect information

Tell us if any listing information is incorrect or outdated.