Unauthenticated RCE in llama.cpp's distributed-inference RPC backend

In brief Most LLM security writing is about prompts. This one is about a network socket. llama.cpp ships an optional RPC backend for spreading inference across several machines. A missing bounds check in the way it parses tensors lets any client that can reach the RPC port read and write the server’s memory, defeat ASLR through protocol-level pointer leaks, and reach remote code execution — no authentication, no prompt, no user interaction. The maintainer advisory (GHSA-j8rj-fmpv-wcxw) was published on March 26, 2026; the fix landed in build b8492. The project’s own docs already said not to expose this service to a network — this is why.

What is this?

llama.cpp is the dominant C/C++ engine for running open-weight models locally, with well over 100,000 GitHub stars. Beyond single-machine use it offers a distributed-inference RPC backend: you build with -DGGML_RPC=ON, start a ggml-rpc-server on each host that owns a GPU, and a main process offloads tensor computations to them over TCP. It is how people run a model that is too big for one box across several.

The flaw is a memory-safety bug in the RPC server’s request parser. It was reported to CERT/CC in February 2026, published as a maintainer security advisory on March 26, 2026, and rated CVSS 3.1 9.8 (Critical), classified as CWE-119 (improper restriction of operations within the bounds of a memory buffer). It affects builds before b8492, where it was patched.

How it works

When the RPC server executes a computation graph, it reconstructs each tensor from bytes on the wire in a helper function. That function only validates the attacker-supplied data pointer when the tensor’s buffer field is non-zero. Set buffer to 0 and the bounds check is skipped entirely, while the data pointer is still taken directly from the request and handed to the compute kernels:

// simplified: buffer = 0 -> nullptr, so the validation block below never runs
result->buffer = reinterpret_cast<ggml_backend_buffer_t>(tensor->buffer);
if (result->buffer) {
    // bounds / sanity checks on result->data live ONLY inside this block
}
result->data = reinterpret_cast<void *>(tensor->data); // taken from the request unconditionally

That single gap gives an attacker an arbitrary read/write primitive against the server process. Because the RPC protocol also hands back raw heap addresses when a client allocates a buffer, there is no need to guess where anything lives: the pointer leaks defeat ASLR, and a function pointer in a buffer’s internal structure can be overwritten to redirect a later request into a memory operation the attacker controls. We are deliberately omitting a working exploit chain — the point here is the class of bug, not a copy-paste weapon. The reporter’s full write-up and the advisory contain the reproduction details for defenders who need them.

Two facts make this practical rather than theoretical. First, no authentication exists — the protocol has none, so reachability equals compromise. Second, the same root cause was behind two out-of-bounds-read flaws patched in this exact function back in 2024; those fixes hardened the tensor-read and tensor-write commands, but the graph-compute command takes a separate code path that was never covered. The underlying weakness has been present since the RPC backend was first merged in May 2024.

Example prompt

The snippet below shows the root cause: the RPC tensor parser skips its bounds check when buffer is 0, so an attacker-controlled data pointer flows straight into the compute kernels. No working exploit is shown — this is for defenders.

# llama.cpp RPC graph-compute RCE (illustrative, defensive)
# The tensor parser only bounds-checks data when buffer != 0:
if tensor.buffer:              # attacker sets buffer = 0 to skip this
    validate(tensor.data)      # [bounds check lives only here]
result.data = tensor.data      # [payload] taken from the wire unconditionally
# Root cause: reachability == compromise; the RPC protocol has no auth.
# Defense: upgrade to build b8492+, bind ggml-rpc-server to 127.0.0.1,
# never publish port 50052, and tunnel nodes over mTLS/WireGuard.

Why it matters

An inference node is a high-value target. It holds model weights, in-flight prompts and often API keys, and it usually sits on a GPU host with broad reach into an internal network. Code execution on one RPC server can mean lateral movement across a cluster, secret and model exfiltration, or simply hijacked GPUs. The default listen port (50052) is easy to fingerprint, and the whole reason the backend exists — inference across machines — pushes operators toward exactly the network exposure that turns this into remote code execution. Docker deployments that publish the port or bind to 0.0.0.0 are directly reachable, frequently as root.

Defenses

• Upgrade to build b8492 or later (the fix, PR #20908), which restores bounds validation on the graph-compute path.
• Never expose the RPC port to an untrusted network. The project’s own RPC README is explicit: the backend is a proof-of-concept, is “fragile and insecure”, and must never run on an open network. Treat it as a trusted-LAN-only service.
• Bind to 127.0.0.1 and tunnel. For genuine multi-host inference, keep every ggml-rpc-server on loopback and connect nodes over a mutually authenticated overlay (WireGuard, mTLS tunnel, private VPC) rather than raw TCP on a routable interface.
• Do not run inference containers with --network=host, and never publish 50052 to the internet. Expose only the intended HTTP API, behind authentication.
• Segment and monitor. Put GPU inference nodes in an isolated network zone; alert on unexpected connections to RPC ports and on inference processes spawning shells.
• Hunt for exposure. Scan your own ranges for reachable RPC ports and confirm none are answering from outside the trusted boundary.

Status

Item	Detail
Reference	CVE-2026-34159 · GHSA-j8rj-fmpv-wcxw · CWE-119
Affected	llama.cpp RPC backend, builds before b8492
Severity	CVSS 3.1 9.8 (Critical) — network, unauthenticated, no user interaction
Reported	CERT/CC, Feb 8 2026 (VU#748698)
Advisory	Published Mar 26, 2026
Fix	Build b8492 (PR #20908)
Prior related	Same function, two out-of-bounds-read fixes in 2024 (CVE-2024-42478 / CVE-2024-42479)
Note	RPC backend is documented as insecure and out of the project’s vulnerability scope