LlamaFactory

What is LlamaFactory?

LlamaFactory (LLaMA-Factory) wraps full fine-tuning, LoRA, QLoRA and other methods for over 100 open models behind a Gradio web UI (LLaMA Board) and a CLI. It covers supervised fine-tuning, preference training (DPO, KTO, ORPO), evaluation, merging and exporting, and can serve the result through an OpenAI-style API with vLLM or SGLang. It needs a GPU for practical training.

Where it fits in a 10–200 person business

• Fine-tune a small open model on your own support tickets, product data or writing style
• Run LoRA/QLoRA training on a single rented GPU instead of sending data to a hosted fine-tuning service
• Export a tuned model and serve it behind an OpenAI-compatible API for internal tools

These use cases are our editorial suggestions, not claims made by the project.

Can a business use LlamaFactory? (license)

This is a plain-English summary of the license text, not legal advice. Third-party models you run with it have their own licenses.

Data & compliance: does LlamaFactory phone home?

Checked against the project's own docs and repository on Oct 29, 2026. "Not documented" means we could not find the answer in official sources; it does not mean the feature is missing or that no data is sent. Third-party services you connect (model APIs, vector databases, search engines) have their own data policies. Compare all apps.

What hardware does LlamaFactory need?

The project does not publish a minimum CPU/RAM figure, so we leave the official value blank rather than guess.

GPU: Required for practical use. The README's estimated GPU memory for a 7B model ranges from about 4-6 GB (2- or 4-bit QLoRA) and 16 GB (16-bit LoRA) to 60-120 GB (full fine-tuning).

The README publishes estimated GPU memory per method and model size, not server minimums. The CUDA Docker image is built on Ubuntu 22.04 (x86_64), CUDA 12.4, Python 3.11 and PyTorch 2.6.0.

Source: github.com/hiyouga/LlamaFactory/blob/main/README.md

How much does it cost to host LlamaFactory per month?

How to deploy LlamaFactory

Method: Docker Compose (build from source, CUDA)

git clone --depth 1 https://github.com/hiyouga/LlamaFactory.git
cd LlamaFactory/docker/docker-cuda/
docker compose up -d
docker compose exec llamafactory bash
# inside the container:
llamafactory-cli webui
# web UI on port 7860, API port 8000

From the official instructions: github.com/hiyouga/LlamaFactory/blob/main/README.md. Always check the source for the current version before you run it.

Known gotchas when self-hosting LlamaFactory

• The Apache-2.0 license covers the framework only. Each base model has its own license (Llama, Gemma, Qwen and others) that governs commercial use of the weights you fine-tune. source
• Docker data volumes are commented out in the Dockerfile by default. Uncomment them or mount hf_cache, shared_data and output, or downloaded models, datasets and trained adapters live only inside the container. source
• The pre-built image targets x86_64 with CUDA 12.4. Ascend NPU and AMD ROCm need their own Docker folders and images. source

Alternatives in Data labeling & vector databases

• Meilisearch Fast search engine with full-text, semantic and hybrid (AI) search, shipped as a single binary.
• Milvus Scalable open-source vector database for semantic search and RAG at large data volumes.
• Qdrant Fast vector database for semantic search and RAG back-ends.
• Chroma Open-source embedding database for vector, full-text and metadata search in RAG apps.
• Label Studio Data labeling tool for text, images, audio and video, used to build and evaluate AI models.
• pgvector Open-source PostgreSQL extension for vector similarity search, next to your relational data.
• Weaviate Open-source vector database with hybrid search, RAG and multi-tenancy built in.

Sources

• GitHub repository: github.com/hiyouga/LlamaFactory
• Official documentation: llamafactory.readthedocs.io/en/latest/
• License text: github.com/hiyouga/LlamaFactory/blob/main/LICENSE
• Hardware requirements, Deploy instructions, Gotcha 1, Gotcha 2, Gotcha 3: github.com/hiyouga/LlamaFactory/blob/main/README.md
• Hetzner Cloud pricing (cost estimate): www.hetzner.com/cloud/regular-performance
• GitHub stars, commits and releases: GitHub REST API, fetched Oct 29, 2026