Core Requirements
| Component | Minimum Recommended | Notes |
|---|---|---|
| RAM | 16GB | Lower if using cloud-based LLM/Embedder APIs. |
| CPU | 8-core CPU | Any modern architecture (including Raspberry Pi for API-only mode). |
| OS / GPU | Windows: GPU (8-12GB+ VRAM) macOS: M-Series Apple Silicon (Intel Macs limited by RAM) Linux: GPU recommended for local execution | Required only for local LLM/embedder execution. |
| Storage | Variable | Dependent on the size of local LLM models stored on disk. |
Component Overhead & Scaling
1. Large Language Model (LLM)
- Cloud/Hosted APIs (e.g., OpenAI): Near-zero local system overhead. Requires API key.
- Local LLMs: High CPU/GPU/VRAM utilization.
- Alternative: Connect AnythingLLM via API to a local LLM hosted on a separate, GPU-equipped network node.
2. Embedder
- Cloud/Hosted APIs: Near-zero local system overhead.
- Local Embedders: Medium-to-high CPU/GPU utilization.
- Alternative: Connect to an external embedder service via API.
3. Vector Database
- Default (LanceDB): Embedded, scales to millions of vectors under recommended core specs.
- External Databases: Near-zero local system overhead.