1. Create a Router
- Navigate to AI Providers → Model Router.
- Click New Router and configure:
- Name: Router identifier.
- Description: Optional reference note.
- Fallback Provider & Model: Default model used when no rules match, and the engine for evaluating LLM-classified rules.
- Cache Cooldown (seconds): Sticky route duration (TTL) for conversation threads. Set to
0to evaluate every message.
2. Rule Types & Syntax
Rules are evaluated sequentially from top to bottom (drag to reorder).
Calculated Rules
Non-LLM evaluation based on message metadata. Execution is instantaneous and free.
- Rule Parameters:
- Property: Prompt Content, Token Count, Message Count, Time of Day, Image Attachment.
- Comparator:
contains,equals,greater than, etc. - Value: Match criteria. (For
contains, use a comma-separated, case-insensitive list. Matches on OR logic for any item in the list). - Route to: Target LLM Provider & Model.
- Logic Operators: Combine multiple conditions using AND / OR toggles.
LLM-Classified Rules
Semantic classification evaluated by the designated fallback model. Incurs one classification LLM call per non-cached evaluation.
- Rule Parameters:
- Match Description: Plain-English definition of the target topic/intent.
- Route to: Target LLM Provider & Model.
3. Deployment
- Open the workspace settings.
- Select the created Model Router from the LLM Provider dropdown.
4. Evaluation Flow & Cache Architecture
For every incoming message, the routing engine executes the following logic:
[Start]
│
▼
[Step 1: Evaluate Calculated Rules] ──(Match Found)──► [Route to Target Model]
│ (No Match)
▼
[Step 2: Check LLM Classify Cache] ──(Cache Hit)────► [Route to Cached Model / Skip]
│ (Cache Miss)
▼
[Perform LLM Classification] ────────(Match Found)──► [Cache Match (5m) & Route]
│ (No Match)
▼
[Cache "No Match" (30s)]
│
▼
[Step 3: Check Sticky Route] ────────(Within Window)─► [Route to Sticky Model]
│ (Expired / None)
▼
[Step 4: Fallback Model]
Caching and TTL Defaults
- Sticky Window (Cooldown): Defaults to 5 minutes (configurable). Restarts on every message that utilizes the active sticky route.
- LLM “Match” Cache: Caches successful semantic matches for the duration of the sticky window (5 minutes).
- LLM “No Match” Cache: Caches negative semantic matches for 30 seconds to prevent classification spam on rapid inputs.