GPUStack

What is GPUStack?

GPUStack manages GPU servers (on-premises, Kubernetes or cloud) as clusters, then deploys models from a catalog by configuring inference engines such as vLLM, SGLang or TensorRT-LLM for you. It adds users, API keys, SSO, usage metering, Grafana/Prometheus monitoring, benchmarking and SSH-accessible GPU instances. It supports NVIDIA and AMD GPUs and several Chinese accelerators (Ascend, Hygon, MetaX and others).

Where it fits in a 10–200 person business

• Turn a few GPU servers into a shared internal model service with per-user API keys
• Run several models across mixed GPUs without hand-tuning vLLM or SGLang flags
• Track token usage and GPU utilization per team for internal chargeback

These use cases are our editorial suggestions, not claims made by the project.

Can a business use GPUStack? (license)

This is a plain-English summary of the license text, not legal advice. Third-party models you run with it have their own licenses.

Data & compliance: does GPUStack phone home?

Checked against the project's own docs and repository on Oct 26, 2026. "Not documented" means we could not find the answer in official sources; it does not mean the feature is missing or that no data is sent. Third-party services you connect (model APIs, vector databases, search engines) have their own data policies. Compare all apps.

What hardware does GPUStack need?

The project does not publish a minimum CPU/RAM figure, so we leave the official value blank rather than guess.

GPU: Required on worker nodes (Linux only): for NVIDIA, a driver supporting CUDA 12.8 or higher plus the NVIDIA Container Toolkit; for AMD, ROCm 7.0 or higher. The GPUStack server itself does not need a GPU.

No CPU, RAM or disk figures are published. Nodes need a modern Linux distribution (AMD64 or ARM64) with NTP installed. The server image bundles PostgreSQL; PostgreSQL 13+ or MySQL 8.0.36+ can be used instead.

Source: docs.gpustack.ai/latest/installation/requirements/

How much does it cost to host GPUStack per month?

How to deploy GPUStack

Method: Docker

sudo docker run -d --name gpustack \
    --restart unless-stopped \
    -p 80:80 \
    --volume gpustack-data:/var/lib/gpustack \
    gpustack/gpustack
# admin password:
sudo docker exec gpustack cat /var/lib/gpustack/initial_admin_password
# then add GPU workers from the Clusters page in the UI

From the official instructions: github.com/gpustack/gpustack#readme. Always check the source for the current version before you run it.

Known gotchas when self-hosting GPUStack

• Installing via PyPI is not supported; use the Docker images. Worker nodes must run Linux (on Windows use WSL2 without Docker Desktop; macOS is not supported). source
• The default admin password is written to /var/lib/gpustack/initial_admin_password inside the server container; set your own with --bootstrap-password. source
• The model catalog, community backends and built-in backend versions update automatically from ota.gpustack.ai. In a restricted network, allow that host, or point --ota-server-url at your own copy. source
• If the NVIDIA GPU Operator is already installed on Kubernetes nodes, disable its device plugin, GPU Feature Discovery, Node Feature Discovery, MIG Manager and CDI components; they conflict with GPUStack. source

Alternatives in Local model runtimes

• Ollama The simplest way to download and run open-weight LLMs locally behind an API. Compare GPUStack vs Ollama
• llama.cpp Lightweight C/C++ engine that runs quantized open models on CPU or GPU, with an OpenAI-compatible server. Compare GPUStack vs llama.cpp
• vLLM High-throughput LLM inference server for GPUs with an OpenAI-compatible API. Compare GPUStack vs vLLM
• LocalAI OpenAI-compatible API for LLMs, vision, voice and images on any hardware. Compare GPUStack vs LocalAI
• Xinference Model-serving platform for LLM, embedding, rerank, speech and image models behind one OpenAI-compatible API. Compare GPUStack vs Xinference

Sources

• GitHub repository: github.com/gpustack/gpustack
• Official documentation, Hardware requirements, Gotcha 1, Gotcha 4: docs.gpustack.ai/latest/installation/requirements/
• License text: github.com/gpustack/gpustack/blob/main/LICENSE
• Deploy instructions: github.com/gpustack/gpustack#readme
• Gotcha 2: docs.gpustack.ai/latest/user-guide/user-management/
• Gotcha 3: docs.gpustack.ai/latest/user-guide/ota-content-updates/
• Hetzner Cloud pricing (cost estimate): www.hetzner.com/cloud/regular-performance
• GitHub stars, commits and releases: GitHub REST API, fetched Oct 26, 2026