## Semantic Router Tuning Reference

### Route design best practices
- 5-20 utterances per route, spanning diverse phrasings
- Include synonyms, abbreviations, and casual forms
- Keep routes mutually distinct; overlapping utterances cause ambiguity
- Add a dedicated `safety`/`jailbreak` route with known attack phrasings

### Threshold tuning (per-route, not global)
```python
thresholds = {
    "greeting": 0.55,      # broad intent -> lower bar
    "refund_request": 0.75, # high-stakes -> higher bar
    "jailbreak": 0.35,      # catch even weak jailbreak attempts
}
```
- Too low: misroutes (false positives)
- Too high: over-misses (falls back too often)
- Tune on a labeled dev set; report precision/recall per route

### Encoder selection
| Encoder | Dims | Notes |
|---------|------|-------|
| text-embedding-3-small | 1536 | API; high quality |
| bge-small-en-v1.5 | 384 | Local; fast; good |
| all-MiniLM-L6-v2 | 384 | Local; fastest; adequate |

### Backend / index choice
| Scale | Backend |
|-------|---------|
| < 1k routes | In-memory numpy (simplest) |
| < 1M routes | Postgres pgvector |
| > 1M routes | Qdrant / Pinecone |

### Hybrid (dense + sparse)
For exact-phrase intents (product names, IDs, emails), add sparse (BM25) vectors.
The `semantic-router[hybrid]` extra provides this. Dense catches paraphrase,
sparse catches exact tokens.

### Latency budget
- Single index query: ~100ms
- Don't chain multiple embedding calls; embed once, compare against all routes
- Cache route vectors; only embed the incoming query at runtime