Open WebUI vs AnythingLLM: A Chat Front End for Ollama or a Document Workspace With RAG Built In

Sora Nguyen

Sora Nguyen

November 19, 2026

Open WebUI vs AnythingLLM: A Chat Front End for Ollama or a Document Workspace With RAG Built In

You have Ollama running, a few models pulled, and the terminal chat is getting old. Maybe you want a browser interface the rest of the household can use, or maybe what you really want is to drop a folder of PDFs, manuals and notes somewhere and ask questions about them. Open WebUI and AnythingLLM both promise to cover that, and both talk to Ollama, but they start from different ends. Open WebUI is a self-hosted chat interface that grew document features. AnythingLLM is a document workspace that grew chat and agent features. Which one fits depends on the job you hand it most often.

Where each project starts from

Open WebUI is a web application you run in Docker (or with pip) and open in a browser. It connects to Ollama and to any OpenAI-compatible API, so the same interface can talk to a local model and a cloud model. Its feature page lists knowledge bases with retrieval, web search with citations, Python tools and functions that run inside the chat, MCP support, and role-based access control with groups. It is built for several users from the start.

AnythingLLM, from Mintplex Labs, comes in two forms: a desktop app for Windows, macOS and Linux, and a Docker server. Everything is organized into workspaces, each with its own documents, settings and chat history. Its README lists a built-in vector database (LanceDB by default), a built-in “AnythingLLM Native Embedder”, agents that can browse the web, a no-code agent builder, MCP compatibility, and support for roughly three dozen LLM providers including Ollama, LM Studio, OpenAI and Anthropic. The desktop app also ships a built-in local LLM engine, so it can run without Ollama at all.

Put simply, Open WebUI treats a chat as the main object and documents as attachments; AnythingLLM treats a workspace of documents as the main object and chat as the way you query it.

Getting each one running next to Ollama

Open WebUI’s quick start gives a single docker run for the common case where Ollama runs on the same host:

docker run -d -p 3000:8080 
  --add-host=host.docker.internal:host-gateway 
  -v open-webui:/app/backend/data 
  -e WEBUI_SECRET_KEY=your-secret-key 
  --name open-webui --restart always 
  ghcr.io/open-webui/open-webui:main

The interface is then at http://localhost:3000. The docs say the container looks for Ollama at http://host.docker.internal:11434; if Ollama lives on another machine, add -e OLLAMA_BASE_URL=.... They also recommend generating the secret key with openssl rand -hex 32 and keeping it stable, or users get logged out whenever the container is recreated. There is an :ollama image tag that bundles Ollama in the same container and a :cuda tag for NVIDIA GPUs.

AnythingLLM’s Docker guide uses a host folder for storage and an .env file:

export STORAGE_LOCATION=$HOME/anythingllm && 
mkdir -p $STORAGE_LOCATION && 
touch "$STORAGE_LOCATION/.env" && 
docker run -d -p 3001:3001 
  --cap-add SYS_ADMIN 
  -v ${STORAGE_LOCATION}:/app/server/storage 
  -v ${STORAGE_LOCATION}/.env:/app/server/.env 
  -e STORAGE_DIR="/app/server/storage" 
  mintplexlabs/anythingllm

It listens on port 3001. The docs warn that without that volume mount “all of your data will be lost when the container is restarted,” and that the container runs as UID 1000, so a storage path under a root-owned folder like /srv needs chown -R 1000:1000 rather than chmod 777. To reach Ollama on the host, AnythingLLM’s guide says to use http://host.docker.internal:11434, add --add-host=host.docker.internal:host-gateway on Linux, and fall back to http://172.17.0.1:11434 if that name does not resolve. Its docs note that localhost networking is behind most Ollama connection problems.

If you only want this on one desktop, AnythingLLM’s installer skips Docker entirely.

A long wooden table in a library reading room with green lamps

Day-to-day chatting with local and cloud models

For plain conversation, Open WebUI is the more complete interface. It is designed to feel like the commercial chat apps: a model picker, chat history in a sidebar, folders, system prompts, model presets, and the option to run the same prompt against more than one model. Because it speaks both the Ollama API and the OpenAI-compatible API, you can put a local model and a hosted one in the same dropdown and switch per conversation. Web search with cited sources is built in.

AnythingLLM can do general chat too, but every conversation lives inside a workspace, and each workspace has a chat mode. Its docs describe three: Agent, which “will automatically use available agent-skills, tools, and MCPs” and is the default for new workspaces in recent versions; Chat, which blends your documents with the model’s general knowledge; and Query, which “only uses information from your uploaded documents” and says so when it finds nothing. That structure is helpful for focused projects and a little heavier if you mostly want quick questions answered.

If you are still choosing the model runner underneath, see Ollama vs LM Studio and llama.cpp vs Ollama.

Asking questions about your own documents

This is where the two are closest in capability and furthest apart in feel.

Open WebUI’s knowledge bases

In Open WebUI you create a knowledge collection, upload files to it, and reference it in a chat or attach it to a custom model. You can also drop a file straight into a single conversation. The feature page says you can “choose between vector search (RAG) for large document sets or full-content injection for precision,” and lists hybrid search that combines “BM25 + vector search with cross-encoder reranking.” Document parsing can be handed to one of several extraction engines, including Apache Tika and Docling, which matters for scanned PDFs and complex layouts. Vectors go into ChromaDB by default, with PGVector also officially maintained and others such as Qdrant and Milvus available as integrations. The environment variable docs list a local Sentence-Transformers model (all-MiniLM-L6) as the default embedder and a default chunk overlap of 100; both can be changed, including pointing embeddings at Ollama.

AnythingLLM’s workspaces

In AnythingLLM you upload documents and then embed them into a workspace. The docs are explicit that “the LLM can only ‘see’ documents embedded in this workspace,” so one workspace per project or topic is the natural pattern. The defaults need no extra services: LanceDB runs inside the app, and the native embedder runs locally. You can switch the vector store to PGVector, Chroma, Qdrant, Milvus, Pinecone, Weaviate or Zilliz, and the embedder to Ollama, LM Studio, OpenAI, Cohere or Voyage AI. Query mode keeps answers tied to your files, which is useful when you would rather get “not found” than a confident guess.

What actually decides answer quality

Neither project publishes benchmarks that settle which retrieves better, and results depend far more on configuration than on the front end. AnythingLLM’s own troubleshooting page explains the mechanics well: documents are split into chunks, the chunks most similar to your question are retrieved (typically four to six), low-scoring chunks are filtered out, and “there is no guarantee that relevant text stays together during this step.” The factors you control in either tool are the same:

  • Chunk size and overlap. Small chunks match precisely but lose context; large chunks keep context but dilute the match.
  • The embedding model. The small defaults are fast on a CPU. A larger embedder served by Ollama often retrieves better, especially for non-English text, but every document must be re-embedded after a switch.
  • The similarity threshold and number of chunks. Too strict and nothing comes back; too loose and the model reads noise.
  • Parsing. A scanned PDF with no text layer produces nothing useful until OCR runs. Open WebUI’s choice of extraction engines is an advantage here.
  • The chat model’s context window. Retrieved chunks have to fit alongside your question and the conversation.

Sharing one server with a household or small team

Open WebUI is multi-user by design. The first account becomes the admin, and the feature page lists “roles, groups, and per-resource permissions,” so you can restrict which models, knowledge bases and tools each group sees. It supports SSO through OIDC, LDAP and SCIM 2.0 provisioning, which matters if you already run an identity provider such as Authentik or Keycloak.

AnythingLLM’s multi-user mode is available only in the Docker version. Its security docs describe three roles: admin with full control, manager who can see all workspaces and most settings but not the LLM, embedder or vector database configuration, and default users who “can send chats only to workspaces they’ve been explicitly added to.” The docs also warn that once multi-user mode is on, you cannot switch back to single-user. The workspace-per-person or workspace-per-team model is easy to explain to non-technical users. The Docker version also includes an embeddable chat widget for putting a workspace on a website.

If your household mainly needs a shared chat front end rather than document workspaces, the comparison of Open WebUI vs LibreChat may be the more relevant one.

Two armchairs facing each other by tall windows

Agents, tools and MCP servers

Both projects now support the Model Context Protocol, so either can call external MCP servers for things like file access or home automation. Beyond that, the approaches differ.

Open WebUI’s extension model is code-first. Tools are Python scripts that the model can call during a chat, edited in a built-in code editor, and functions can “filter, transform, or route every message.” It can also auto-discover tools from OpenAPI servers, and the separate Pipelines project runs heavier custom logic in its own container. Because tools execute Python on your server, only install ones you have read or trust, and limit who can add them.

AnythingLLM’s agents are configured more than coded. The README lists web browsing, a no-code agent builder, custom agents, scheduled tasks and memories. Agent mode depends on the chosen model supporting native tool calling, so small local models may struggle to use tools reliably in either app.

Licensing: Open WebUI’s branding clause versus AnythingLLM’s MIT

AnythingLLM is MIT licensed, and its Docker version lists custom themes and white-labeling among its features.

Open WebUI changed its license with v0.6.6 in April 2025. Code up to v0.6.5 remains BSD-3-Clause. From v0.6.6, the license adds a branding clause: you “may NOT alter, remove, or obscure any ‘Open WebUI’ branding” unless the deployment has 50 or fewer users in any 30-day window, you have written permission as a substantive contributor, or you hold an enterprise license that allows it. The project’s own license page states that v0.6.6 and later is “not an OSI-approved ‘open source’ license.” For a home server you are well under the threshold and nothing changes. For a company deployment with more than 50 users that wants its own logo, it does.

Telemetry and what leaves your machine

AnythingLLM collects anonymous usage telemetry through PostHog by default. Its README lists what is sent: installation type, document added or removed events without document details, vector database type, LLM provider and model tag, and a “chat sent” event without the content, with “no IP or other identifying information” collected. To opt out, set DISABLE_TELEMETRY="true" in the server or Docker .env file, or turn it off under Privacy in the app’s sidebar.

Open WebUI’s example environment file sets SCARF_NO_ANALYTICS=true, DO_NOT_TRACK=true and ANONYMIZED_TELEMETRY=false, variables that switch off analytics in bundled libraries. If privacy is the reason you self-host, setting these explicitly in your compose file is a reasonable habit.

In both tools, your prompts and documents leave the machine only when you configure a cloud provider for the chat model or the embedder. A fully local setup with Ollama and the built-in embedder keeps everything on your hardware.

Hardware: where embedding and inference actually run

Neither front end needs much on its own. AnythingLLM’s Docker guide suggests at least 2 GB of RAM and 10 GB of disk for cloud hosting. The heavy work happens in the model runner, so size the machine for Ollama and the models you plan to use.

The one exception is embedding. Both apps embed documents locally by default, on the CPU, which is fine for a few hundred pages but slow for a large archive. Open WebUI’s :cuda image moves its local embedding, reranking and speech-to-text models to the GPU; the docs note that this does not give Ollama a GPU, which has to be configured separately. In either app you can instead point embeddings at Ollama, which then uses whatever GPU it has. If several people will hit the same server at once, the runner matters more than the front end; the Ollama vs vLLM comparison covers that case.

Picking by the job you need done

  • A shared chat interface for several people, local and cloud models side by side: Open WebUI. Its multi-user model, groups and SSO are more complete, and the chat experience is its core.
  • A private assistant over one set of documents on a single desktop: AnythingLLM desktop. Install, pick Ollama or the built-in engine, create a workspace, drop in files.
  • Separate document collections for different people or projects on a server: AnythingLLM in Docker with multi-user mode, where workspace membership controls who sees what.
  • Heavy document parsing, scanned PDFs, hybrid search and reranking: Open WebUI, for its extraction engine choices and retrieval options.
  • Custom Python logic in the chat loop: Open WebUI tools and functions.
  • A rebranded deployment for more than 50 users: AnythingLLM’s MIT license, or an Open WebUI enterprise license.

Both are free to run and both point at the same Ollama endpoint, so nothing stops you from running them side by side for a week with the same documents and the same models before committing.

More articles for you