PrivateGPT

What is PrivateGPT?

PrivateGPT provides an API (with a demo UI) for ingesting files and asking questions about them, using an OpenAI-compatible model server that you run separately, such as Ollama, LM Studio, llama.cpp or vLLM. Developers are expected to build their own apps on the API.

Where it fits in a 10–200 person business

• Back-end for an internal document assistant where no data may leave the network
• Pilot or demo of private document Q&A
• Base for a custom app that needs ingestion and retrieval APIs

These use cases are our editorial suggestions, not claims made by the project.

Can a business use PrivateGPT? (license)

This is a plain-English summary of the license text, not legal advice. Third-party models you run with it have their own licenses.

What hardware does PrivateGPT need?

The project does not publish a minimum CPU/RAM figure, so we leave it blank rather than guess.

No official figures for PrivateGPT itself. The model server dominates; the quickstart's example LLM (qwen3.5:35b) is about 24 GB.

Source: docs.privategpt.dev/getting-started/quickstart

How much does it cost to host PrivateGPT per month?

How to deploy PrivateGPT

Method: uv (Python) or Docker

uv tool install --python 3.11 \
  --find-links https://wheels.privategpt.dev/packages/ "private-gpt[core]"
OPENAI_API_BASE=http://localhost:<llm-port>/v1 \
OPENAI_EMBEDDING_API_BASE=http://localhost:<embedding-port>/v1 \
private-gpt serve

From the official instructions: github.com/zylon-ai/private-gpt#readme. Always check the source for the current version before you run it.

Known gotchas when self-hosting PrivateGPT

• Requires a separately running OpenAI-compatible LLM server. PrivateGPT does not run models itself. source
• Ollama has no tokenizer endpoint, so with Ollama PrivateGPT falls back to approximate token counting, which can affect context-window management. source

Alternatives in Knowledge base & RAG

• RAGFlow RAG engine with deep document parsing (OCR, layout and tables) and cited answers. Compare PrivateGPT vs RAGFlow
• AnythingLLM All-in-one private chat with your documents, agents and an embeddable widget. Compare AnythingLLM vs PrivateGPT
• Onyx (formerly Danswer) AI chat and enterprise search connected to 50+ workplace apps. Compare Onyx (formerly Danswer) vs PrivateGPT
• FastGPT Knowledge-base Q&A and visual AI workflow platform. Compare FastGPT vs PrivateGPT
• MaxKB Enterprise knowledge-base and agent platform with RAG and workflows. Compare MaxKB vs PrivateGPT

Sources

• GitHub repository: github.com/zylon-ai/private-gpt
• Official documentation, Hardware requirements, Gotcha 2: docs.privategpt.dev/getting-started/quickstart
• License text: github.com/zylon-ai/private-gpt/blob/main/LICENSE
• Deploy instructions, Gotcha 1: github.com/zylon-ai/private-gpt#readme
• Hetzner Cloud pricing (cost estimate): www.hetzner.com/cloud/regular-performance
• GitHub stars, commits and releases: GitHub REST API, fetched Oct 22, 2026