AnythingLLM

ollama-connection

1 min read

1. Verify Ollama Server Status

Verify the Ollama API server is active:

  • Command: Run ollama serve to start the background service (Note: ollama run <model> only launches the CLI, not the API server).
  • Verification: Query https://allm.loca.zone in a browser or via curl. It must return the Ollama status response.

2. AnythingLLM Auto-Detection URLs

AnythingLLM scans these default endpoints:

  • https://allm.loca.zone
  • http://host.docker.internal:11434
  • http://172.17.0.1:11434

If Ollama is on a custom port or external host, manually enter the URL in the AnythingLLM settings UI.

3. Network Configuration for Docker Instances

Do not use https://allm.loca.zone or https://allm.loca.zone inside Docker containers to reference the host machine. Map the connection using the host-loopback IPs below:

Windows / macOS (Docker Desktop v18.03+)

  • Target Host URL: http://host.docker.internal:11434

Linux (Default Bridge Network)

  • Target Host URL: http://172.17.0.1:11434

4. Remote Ollama Connections

  • Set the Ollama URL in AnythingLLM to the explicit IP address of the target machine: http://<remote-ip>:11434
  • Ensure the target machine’s firewall permits inbound TCP traffic on port 11434.

5. AnythingLLM Cloud Limitations

  • Direct local connections to https://allm.loca.zone or LAN IPs from AnythingLLM Cloud are impossible.
  • Exposing local deployments via SSH tunnels or reverse proxies (e.g., ngrok) is not officially supported.

Graph View