AnythingLLM

Turn local documents into cited conversations with replaceable model and vector layers

AnythingLLM can process formats such as PDF, TXT, and DOCX through its collector and embed them into workspaces for retrieval-augmented conversations. LanceDB is the default vector database, with alternatives including PGVector, Pinecone, Chroma, Weaviate, Qdrant, and Milvus. Model and embedding choices are similarly broad, covering services and runtimes such as OpenAI-compatible APIs, Ollama, LM Studio, LocalAI, Gemini, Anthropic, Bedrock, and OpenRouter. This supports both predominantly local deployments and configurations that deliberately combine external providers behind the same workspace UI.

Sources: [1]

Add agents, MCP, no-code flows, and scheduled jobs

AnythingLLM includes workspace agents, a no-code Agent Flow builder, MCP compatibility, and scheduled tasks that can run recurring prompts with agent capabilities. These features turn the product from a document-chat surface into a place where tools and repeatable workflows can execute. Its Intelligent Skill Selection feature is designed to avoid injecting every available tool into every prompt, selecting relevant skills for the request so large tool catalogs can be used with lower prompt overhead.

Sources: [1][3]

Route conversations across models while preserving longer-lived memory

Dynamic Model Routing lets administrators define rules that send a conversation to a selected provider and model. The implementation evaluates calculated rules and LLM-based rules before instantiating the delegated model provider. Automatic and user-managed memory adds a separate layer for carrying important user or workspace information across conversations. Together, these features suit deployments that want to reserve expensive models for specific cases instead of binding every request to one provider while still preserving useful long-term context.

Sources: [1][2]

For teams combining RAG, local models, routing, and agents

For an individual, AnythingLLM provides a practical way to pair local runtimes such as Ollama or LM Studio with private documents from a desktop application. For teams, the Docker edition can consolidate internal-document Q&A, workspace separation, agent workflows, scheduled automation, and an embeddable website chat interface. It is particularly relevant when the ability to replace the underlying model or vector database later is more important than optimizing around one vendor's ecosystem.

Sources: [1]

Desktop and Docker differ, and privacy depends on connected providers

The README explicitly limits multi-user instances, permissioning, and the embeddable web chat widget to the Docker edition. Desktop is optimized for easy personal local use, while team or public-service deployments should evaluate the server edition and its operational requirements. The monorepo also contains a React/Vite frontend, Node.js Express server, and document collector, so production use still requires deliberate persistence, backup, and availability planning for whichever vector databases and model providers are selected.

AnythingLLM includes anonymous usage telemetry that can be disabled with DISABLE_TELEMETRY=true or from the application settings. Disabling telemetry does not eliminate outbound traffic when external LLMs, embedding services, vector databases, or tools are configured, so self-hosting should not be treated as synonymous with fully local data processing. The core repository is distributed under the MIT License.

Sources: [1][4]