Ollama on 0.0.0.0, and other ways to give away your GPU

Ollama listens on 127.0.0.1:11434 by default. Only processes on the same machine can reach it, so it does not bother with authentication. That is a reasonable default for a tool whose main job is “run a model on my laptop.”

The trouble starts the first time something else needs to reach it. A web UI in a Docker container, a script on another box, a teammate’s machine. The docs say to set OLLAMA_HOST, so you do:

OLLAMA_HOST=0.0.0.0 ollama serve

Now the API is bound to every interface on the machine, still with no authentication, because Ollama does not have any to turn on. As of this writing there is no password, no API key, and no allowlist in the server itself. If the machine has a public address, or sits in a cloud VPC with a permissive security group, the API is on the internet.

Internet-wide scans turn up thousands of these. Most are home boxes and hobby servers. Some are companies.

What a stranger can do with it

Everything the API can do, since there is nothing between them and it.

• GET /api/tags lists every model on the box, which is a decent fingerprint of what the machine is for.
• POST /api/generate and /api/chat run inference on your GPU with their prompts. On a cloud instance this shows up as a bill.
• POST /api/pull downloads any model from the registry onto your disk until the disk is full.
• DELETE /api/delete removes models.
• POST /api/create builds a model from a Modelfile. Older versions had a path traversal in the pull path (CVE-2024-37032, fixed in 0.1.34) where a caller could point the server at a hostile registry and write files outside the model directory. Exposed instances tend to be old instances.

None of this is an exploit. It is the documented API, reached by someone it was not meant for.

It is not just Ollama

The same shape shows up across the local-inference ecosystem. The tools assume a trusted network and leave auth to you.

Server	Default bind	Built-in auth	The thing people flip
Ollama	127.0.0.1	None	OLLAMA_HOST=0.0.0.0
llama.cpp llama-server	127.0.0.1	--api-key, off by default	--host 0.0.0.0
vLLM	all interfaces	--api-key, off by default	Nothing. It ships open.
LM Studio	localhost	None	“Serve on local network” toggle
Open WebUI	all interfaces	Login, on by default	WEBUI_AUTH=False
LiteLLM proxy	all interfaces	master_key, optional	Leaving master_key unset

Docker makes it worse in a way that surprises people. -p 11434:11434 publishes the port on every host interface, and Docker writes its own iptables rules that sit in front of ufw, so a host firewall that “blocks everything except 22 and 443” does not block it. Compose files copied from a README almost always do this:

services:
  ollama:
    image: ollama/ollama
    ports:
      - "11434:11434"   # every interface on the host, bypasses ufw

Bind it to loopback explicitly. Anything else that needs it can join the same compose network and talk to it by service name.

services:
  ollama:
    image: ollama/ollama
    ports:
      - "127.0.0.1:11434:11434"

Putting something in front of it

Keep the model server on loopback or a private Docker network. Then pick one of these for whatever actually needs to reach it.

A reverse proxy with auth. Caddy is the least typing:

llm.example.com {
    basic_auth {
        alice $2a$14$...bcrypt hash here...
    }
    reverse_proxy 127.0.0.1:11434
}

That gets you TLS and a password. Swap basic_auth for forward_auth against an OIDC provider once it is more than one person.

A tunnel with an identity check. Cloudflare Tunnel with an Access policy, or Tailscale, both let you reach the server from anywhere without opening a port. Nothing listens on the public interface at all, which is the state you want to be in.

A private network and nothing else. If every client is inside the VPC, bind to the private interface and let the security group do the work. Then read the security group. “Allow all inbound from 0.0.0.0/0” is how most of the exposed instances got that way.

Checking your own machines

From the box itself:

ss -ltnp | grep -E '11434|8000|8080|1234'

Anything showing 0.0.0.0: or *: is bound to every interface. From somewhere else:

curl -s http://YOUR_PUBLIC_IP:11434/api/tags

If that returns JSON, you are on the list. Add the port to whatever external scan you already run, because the next person to set OLLAMA_HOST will not tell you.

Binding a service to all interfaces without auth is an old mistake. The only new part is the GPU behind it, which is why people go looking.